"the Fed" generally refers to the Federal Reserve Board, an American financial institution. When referring to the FBI, we say "FBI". The agents of the FBI are popularly knows as "feds" (no initial capital letter).
So: The fed(s) let Swagger off the hook after they arrived at the Department of Justice.