Could it be as simple as Peter having known where Kate was, but wanting to keep Neal away from her for Neal's own good? Perhaps Kate isn't as in love with Neal as he is with her, and she wants to get her hands on his hidden stash (which, coincidentaly, OPM also seems to want). She doesn't actually seem to be held captive based on what we've seen of her - what if she's lying and trying to con Neal? Fowler is the one who set Neal up, but it seems he did it to get to Peter. Why would he want to do that? Maybe he wanted Peter to lead him to Kate so that they could use her to get whatever Neal has.

What I really can't figure out is why Peter was wearing the ring. Did Neal find the picture of Peter finding Kate? Is it an FBI ring that all agents have? It is a red herring? Wtf is it?!