The Secretary may (for such period, or indefinitely, as he deems necessary to effectuate the purposes of this chapter) refuse to provide, or withdraw, inspection service under subchapter I of this chapter with respect to any establishment if he determines, after opportunity for a hearing is accorded to the applicant for, or recipient of, such service, that such applicant or recipient is unfit to engage in any business requiring inspection under subchapter I because the applicant or recipient, or anyone responsibly connected with the applicant or recipient, has been convicted, in any Federal or State court, of (1) any felony, or (2) more than one violation of any law, other than a felon