You cannot legally refuse visitation rights - not unless a court agrees with your assessment of the situation, finds the father to be an unfit parent, and makes a ruling to that effect. However, a lot of what you have stated are reasons for your attitude are merely observations and (current) personal preferences - surely, when you got involved with the father, you knew he smoked and drank and swore? To object to that kind of behaviour now is more than just a little two-faced and insincere.
As for the rest: advising you on the basis of what you have decided to 'share' and without access to that which you have decided to withhold, would be a waste of time. You need professional help. Which doesn't mean a friend who works as a secretary for a lawyer. And try finding a mediator who does their job properly - a mediator is there to help resolve disputes between parties: they are not there to make psychological or sociological assessments of any of the parties involved in the dispute, and impart such to the other party/parties. A mediator is SUPPOSED to be completely impartial and professional, to deal only with the issues in dispute.
There is no such legislation as you suggested in your first paragraph, that would allow you to withhold visitation rights. Further, maintenance and access are two completely unrelated and independent issues in terms of the Children's Act - withhold access at your own peril. If you withhold access without legal grounds and the support of the courts, you could be in very deep, very hot water. Get proper, professional legal advice. Failure to do so may just land you up in a jail cell, doing time, and your child living permanently with his father and his family. Is that really what you want?
At the same time as you solicit that legal advice, apply for maintenance for your child. Just remember that the maintenance is for the child's (verifiable) costs - not yours. And whatever you're awarded will be 'proportional' - i.e. if you earn double what he earns, you will pay double what he pays towards your son's costs. Costs - not luxuries. Nothing but basic costs. Of course, if he's unemployed, you will most probably get nothing - but if the ruling is in place and he finds employment, you will be entitled to claim against his income.
p.s. what is "Double Airway infection"? The closest I can get to an explanation is "double pneumonia" which is an infection of both lungs - but that's not the airway itself. The airway is what we used to call the 'windpipe' - and there's only one. Any infection of the lungs in a small child is serious - but only to the degree where you honestly believe that the father would not exercise due care and diligence with regard to the child. Children fall all the time, get bruised, eat different things that adults wouldn't (mud, for instance!) - that is what they are SUPPOSED to do: they're learning about their world, trying things, tasting things, feeling things. It's NORMAL!
p.p.s. children and dogs have a natural affinity to each other - and so far, this hasn't been outlawed. Exposure to animals from an early age generally bodes well for the immune system of a child, and lack thereof is thought to cause allergies in older children when exposed to animals. In fact, the conclusion of several studies such as this one (http://ukpmc.ac.uk/abstract/MED/10231320) has been that "Pet exposure during the first year of life and increasing number of siblings were both associated with a lower prevalence of allergic rhinitis and asthma in school children." - so what's wrong with a little dog-hair?