Acutally ddawg, four wires are being used. The black and yellow are your ground.
While black and yellow (or more often just yellow) may be tied to ground in some installations, this is by no means standard, and is not required for telephone operation. Very old installations may only have 1 pair of wires (red/green) and not even have the second pair (yellow/black.)
In the 1960s it was popular to use the black/yellow pair to provide low voltage power (24VAC) for lighted phone dials--there would a transformer somewhere in the house plugged into a wall outlet and connected to the black/yellow pair.
In the 1990s is was popular to use the black/yellow pair for a second phone line.
To extend an existing single-line installation, you can start at any existing working jack, the demarc point, or any place where the wires are accessible for splicing and run a minimum of 1 pair to the new location. You can still buy standard 2-pair phone wire (red/green/black/yellow) which is fine for extending a 1920s vintage service. The newer Cat3 and Cat5 twisted pair wiring typically has 4 pairs of wires white-blue/blue-white, white-orange/orange-white, white-green/green-white, and white-brown/brown-white.
It is good to maintain polarity (red to red) if possible, although most phones will operate even if the wires are reversed. Telephone service uses 48VDC except when the phone rings, which is 90VAC. To avoid getting zapped while making connections, take a phone somewhere else in the house off the hook while working on the telephone wiring.
If you have been upgraded to a modern style demarc point on the outside of the house, open the gray box and you can unplug the whole house from the phone comapny while you are working on the wiring.
In a 1920s house, a typical demark would be mounted on the basement ceiling near where the phone wire enters the house. It would have two threaded terminals with nuts (probably takes a deep 7/16" socket) where the red & green wires running to the jack(s) connect.