We need more information...
Is this a totally separate service from the house, or is the house and the new building fed from the same service? Is this a subpanel, connected to a main panel somewhere else? Since you're talking about running wire from a meter on a pole it sounds like a whole new thing.
Where is the main service disconnect? Is it a separate switch at the pole, or is it going to be the main breaker in the panel at the new building? The ground wire (technically it's called the EGC, or equipment grounding conductor) is connected at the main service disconnect, wherever that is.
For a 100 amp service, typically a #6 wire is adequate for the EGC. Is the building built yet, or is it still just a bare patch of ground? Depending on the codes in your area, you may be required to use what's called an Ufer ground instead of ground rods - basically, it's a piece of rebar 20 feet long, laid horizontally inside the concrete footing or slab. This method is called out in the 2005 edition of the National Electric Code; some parts of the country still use earlier versions that may not include the Ufer ground as a requirement, and then they probably want you to have two ground rods in the earth next to the building, five or ten feet apart (I don't have my code book in front of me so I can't be sure on the exact details.)
My guess (and this is just a guess so don't take it as gospel) you have a new service from a new meter on a pole that's separate from your house or any other existing buildings, and you're bringing the service conductors from the pole into the new building. Inside the building there will be a new panelboard, where the main circuit breaker is. If this is the case, the EGC wire is connected inside this new panelboard, and then is connected to either an Ufer ground connection in the building's footing or to a couple ground rods.
If you're already working with a local electrician, ask him about the grounding requirements to make sure you do the right thing. There are lots of variables here and it's difficult to give an absolute answer over the Internet. He can see just what your situation is in person, and will know what the local codes are.