Hot-plugging audio equipment

Can audio equipment (samplers, synths, etc) be plugged into a mixer or audio interface (or have equipment plugged into them) while all the equipment is turned on? If yes, is this common?

Hot-plugging audio equipment