If they stop by unannounced, you still have your forth amendment rights. They're not allowed in without a warrant. § 178.23(c)(2) allows the C&R holder to elect to have the annual inspection carried out at the nearest ATF office.
I have personally never heard of any C&R holder ever having an annual inspection. It would usually be a waste of time unless they actually suspected you of criminal activity or one of your guns having been involved in such, at which point they will likely have a warrant. They have bigger fish to fry than making sure C&R holders dotted all the 'i's in their bound book.
I just read that section after I read somewhere that you lose warrant protection if you have a C&R.
So here is the text.
"(c) Any ATF officer, without having
reasonable cause to believe a violation
of the Act has occurred or that evidence
of the violation may be found
and without demonstrating such cause
before a Federal magistrate or obtaining
from the magistrate a warrant authorizing
entry, may enter during
hours of operation the premises, including
places of storage, of any licensed
collector for the purpose of inspecting
or examining the records, documents,
firearms, and ammunition referred
to in paragraph (a) of this section
(1) for ensuring compliance with
the recordkeeping requirements of this
part not more than once during any 12-
month period or (2) when such inspection
or examination may be required
for determining the disposition of one
or more particular firearms in the
course of a bona fide criminal investigation.
At the election of the licensed
collector, the annual inspection permitted
by this paragraph shall be performed
at the ATF office responsible
for conducting such inspection in closest
proximity to the collectors premises"
If I am reading that correctly, they can enter without a warrant if its part of a criminal investigation?