Pygmy sperm whales have a small, compact body with a small and rounded dorsal fin. Each individual whale’s dorsal fin is a slightly different shape. While on the water’s surface, pygmy sperm whales have a low profile because their head and back are somewhat flat. Their head is sometime described as shark-like because of their pointed snout and narrow lower jaw.

Pygmy sperm whales have wrinkled skin and a brown to blue-gray back. Their underside is paler with white or pink tones. They do not have teeth in their upper jaw but have 10 to 16 pairs of teeth in the lower jaw. Their eyes are dark and bulging, and they have a marking behind the eye that is often called a false gill because it looks like a fish's gill cover.

Pygmy sperm whales can sometimes be confused with dwarf sperm whales, their closest relative. These two types of whales were not distinguished as separate species until 1966. In the wild, it is very difficult to distinguish between them because they have similar appearances and geographic ranges.