It depends on what the siding material is. Cracks are a telltale sign of siding damage but discoloration, dents, and missing pieces may be signs of siding damage.
Generally, a home’s siding should last for several decades. Damage from covered perils cuts siding’s longevity short, and is often grounds for replacement.