Many of young Edmund's forefathers fought in the French & Indian Wars and the American Revolution. His grandfather, Ensign Edmund Rice was a veteran of the War of 1812. His father, Moses M. Rice was a prosperous businessman who was a pioneer in the early horse railroads as well as steam. He was also involved in real estate, mercantile endeavors and helped develop theCambridge Gasworks and Cambridge Waterworks.

Edmund grew up on Brattle Street in Cambridge, not far from the present day campus of Cambridge University. He was the fourth born of seven children of this well to do family and was educated at the local schools. In 1856 he entered Norwich University in Vermont, where he became a cadet at the nation's second oldest Military Academy. He remained nearly three years when he yearned for adventure and had his father arrange for him to become a captain's apprentice on the clipper ship, Snow Squall.

The Snow Squall set sail from Long Wharf, Boston in September of 1858 for Shanghai, China. His first experience in action was in driving off pirates who attacked the ship while it was becalmed in the China Sea. Since he was educated in military and artillery matters at Norwich University, he was given charge of one of the small bore swivel cannons which were mounted on the ship. He was just 16 years of age when he loaded the cannon with lead pipe and nails and repulsed the attack of the Asian pirates.