You don't get access to the national health system. Some people get private health insurance (usually an international health insurance), others pay themselves a symbolic salary to qualify. Some go without any health coverage at all.
I've heard people run into some issues with the local banks when taking out
loans, if they can't demonstrate social insurance contributions. AFAIK, the banks can usually be convinced upon seeing proof of stable
dividends payments.
Haven't heard of any other issues.
That's if you're a one-person business. If there are multiple people, the authorities may find it suspicious if everyone is a consultant and not an employee. If you're going to build up a team in Cyprus, consider hiring them as employees. Payroll taxes are surprisingly high in Cyprus, so take that into account.