Asbestos fibres tend to split longitudinally to create nanoscale-wide spears that cannot be broken down by the body's defences, and can cause mesothelioma, especially to people with workplace exposure.
This "nano" is a definite "no-no".
Depending on the application, carbon nanotubes may come as long tubes ("long" compared to the size of a cell), and so there is some geometrical similarity to asbestos, but the chemical behaviour in the body is radically different, and environmental breakdown processes will be quite different. Carbon nanotubes should be separately assessed for health hazards, and not grouped with "asbestos". It may be considered "safe" as long as it is in a form that cannot be ingested.
Every nanomaterial should be considered on its own merits - perhaps to the extent of determining at what scale the behaviour deviates significantly from the Material Safety Data Sheet (MSDS) for the "macroscale" material.