It is not mandatory that you have a signed letter from the other parent, as long as the border officials are satisfied that you have proper approval from that parent, or a court, to travel with the child.
It's important that you give your co-parent notice and preferably get their express permission in writing if you wish to travel outside the state with your child. If the other parent refuses to cooperate, you can ask for the court's permission in advance.
Unless your specific court order prevents out-of-state travel or other restrictions, generally each parent may choose to travel.
Your child will need a Child Travel Consent any time they're traveling without all their parents or legal guardians present. If your child is traveling alone, with only one parent, or with a non-legal guardian, they will require a Child Travel Consent signed by both parents or legal guardians.
The United States Does Not Have Exit Controls: Two-parent consent is not always needed for a minor to leave the United States. Without a valid court order prohibiting the child's travel outside of the United States, there may be no way to prevent an abduction.
Many states have their own move-away laws regarding child custody situations. In California, if your ex-spouse has sole custody, you can only prevent him or her from taking your child out of the country if you can prove the move will harm your child.
In California, when there is no court order regarding custody in place, both parents have equal rights to the children. However, California law requires written consent from the other parent before removing minor children from the state, whether for relocation or vacations.
If your child is traveling alone, with only one parent, or with a non-legal guardian, they will require a Child Travel Consent signed by both parents or legal guardians. A Child Travel Consent is sometimes called a letter of permission to travel.
In the USA, though, unless there's a court order requiring permission from both parents, a child may travel (even to overseas) with either parent/guardian without showing proof of permission from the other parent.
The US government places no restrictions or requirements on you when exiting or entering the country with your child. Many foreign governments might require require documentation for entry or international air carriers to board their plane.