Set in 1926, against the backdrop of the tumultuous Chinese revolution, the story revolves around an American gunboat, the USS San Pablo, patrolling the Yangtze River. This gunboat, nicknamed the "Sand Pebble," has a new engineer, Jake Holman, who prefers working directly on the engines rather than delegating. Holman's unconventional ways and individualistic approach immediately put him at odds with his shipmates and the existing naval practices. His technical proficiency and dedication to the engine room earn him respect, yet his disdain for authority complicates his position on the ship. The narrative explores Holman's internal struggle with military protocol and his efforts to navigate the complexities of his new environment.

As the political situation in China intensifies, the San Pablo's mission becomes ever more perilous. The crew finds themselves in a quagmire of diplomatic tension, caught between their orders from distant leaders and the reality of the nationalist fervor sweeping across China. Amidst this, Holman forms a bond with a local missionary schoolteacher and befriends a Chinese coolie working on the boat. These relationships deepen his connection to the country and its people, further complicating his feelings towards the ongoing conflict. The crew's loyalty and unity are tested as they are drawn into the violent struggle, forcing them to confront their roles as foreigners in a country fighting for its sovereignty.