The novella's plotline is heavily based on God Sees the Truth, But Waits by Leo Tolstoy. King states that the story was inspired by various prison films he watched growing up.
Andy Dufresne, a banker from Maine, is arrested for the double murder of his philandering wife and her lover. He is sent to Shawshank Prison for life. At the prison, he meets Red, a prisoner who specializes in procuring items from the outside world.
As a free man, Andy had been a rockhound, so he asks Red to get him a rock hammer, a tool he uses to shape the rocks he finds in the exercise yard into small sculptures. One of the next items he orders from Red is a large poster of Rita Hayworth. When taking the order, Red reflects that Andy is uncharacteristically excited about the poster, but does not think more of it at the time.
One spring day, Andy and Red and some other prisoners are tarring a roof when Andy overhears a particularly nasty guard griping over the amount of tax he will have to pay on a sum of money bequeathed to him by a long-estranged brother. Andy approaches the guard, almost getting thrown off the roof in the process, and tells him that he can legally shelter the money from taxation by giving it to his wife.
A gang of sexually aggressive prisoners called "The Sisters," led by Bogs Diamond, rapes any prisoner they feel they can handle, and Andy is no exception. However, when Andy makes himself useful to the guards, they protect him from "The Sisters." One night, Bogs is found in his cell 'inexplicably' unconscious and severely beaten. Andy is also allowed to stay alone in his cell instead of having a cellmate like most other prisoners. For a short period, he shares a cell with an Indian called Normaden, but is soon alone again, Normaden having complained about a "bad draft" in the cell.
Andy's work assignment is shifted from the laundry to the prison's small library, previously under the stewardship of Brooks Hatlen, one of the few other prisoners with a college degree. Red dryly notes that Brooks' degree is in animal husbandry, "but beggars can't be choosers." The new assignment also allows Andy to spend more time doing financial paperwork for the staff. Andy applies to the Maine State Senate for funding to expand the library. For years he gets no response to his weekly letters until the Senate finally sends him $200, thinking Andy will stop requesting funds. Instead of ceasing his letter writing, he starts writing twice as often. His diligent work results in a major expansion of the library's collection, and he also helps a number of prisoners earn equivalency diplomas, preparing them for life after parole.
The warden of Shawshank, Norton, also realizes that a man of Andy's skills is useful. He has started a program called "Inside-Out" where convicts do work outside the prison for slave wages. Normal companies outside cannot compete with the cost of Inside-Out workers, so they offer Norton bribes not to bid for contracts. This cash has to be laundered somehow, and Andy makes himself useful here as well.
One day, Andy hears from another prisoner, Tommy Williams, whose former cellmate had bragged about killing a rich golfer and a lawyer's wife (Andy latches onto the idea that the word "lawyer" could easily have been mixed up with "banker," the professions being similarly viewed by the uneducated public), and framing the lawyer for the crime. Upon hearing Tommy's story, Andy realizes that if this evidence could be brought before a court, he could be given a new trial and a chance at freedom. Norton scoffs at the story, however, and as soon as possible he makes sure Tommy is moved to another, lower security prison, presumably as compensation for promising that he never talks about this anymore. Andy is too useful to Norton to be allowed to go free; furthermore, he knows details about Norton's corrupt dealings. After losing his customary cool with the warden over the issue, and spending a couple of months in solitary confinement as a result, Andy resigns himself to the fact that the prospect for his legal vindication has become non-existent.
Before he was sentenced to life, Andy managed, with the help of his closest friend, to sell off his assets and invest the proceeds under a pseudonym. This alias, Peter Stevens, has a driver's license, Social Security card and other credentials. The documents required to claim Stevens' assets and assume his identity are in a safe deposit box in a Portland bank; the key to the box is hidden under a black volcanic rock along a rock wall lining a hay field in the small town of Buxton, not far from Shawshank.
After eighteen years in prison, Andy shares the information with Red, describing exactly how to find the place and how one day "Peter Stevens" will own a small seaside resort hotel in Zihuatanejo, Mexico. Andy also tells Red that he could use a man who knows how to get things. Red, confused about why Andy has confided this information in him, reflects on Andy's continued ability to surprise.
One morning, after he has been incarcerated for nearly 27 years, Andy disappears from his locked cell. After searching the prison grounds and surrounding area without finding any sign of him, the warden looks in Andy's cell and discovers that the current poster pasted to his wall—of a young Rita Hayworth (Raquel Welch in the film adaptation of the novella)—covers a man-sized hole. Andy had used his rock hammer—and a replacement when the original wore down—not just to shape rocks, but to carve a hole through the wall. Once through the wall, he broke into a sewage pipe, crawled through it for some 500 yards, emerged into a field beyond the prison's outer perimeter and vanished. His prison uniform is found two miles away from the outfall. How he made good his escape with no equipment, clothing or known accomplices, nobody can determine.
A few weeks later, Red gets a blank postcard from a small Texas town near the Mexican border, and surmises that Andy crossed the border there. Shortly afterwards, Red is paroled. After nearly 40 years' imprisonment, he finds the transition to life "outside" a difficult process. On the weekends, he hitchhikes to Buxton, searching for suitable hay fields from Andy's "directions". After several months of wandering the rural town roads, he finds a field with a rock wall on the correct side, with a black rock in it. Under this rock, he finds a letter addressed to him from "Peter Stevens" inviting him to join Peter in Mexico. With the letter are 20 $50 bills. The story ends with Red violating his parole to follow Andy to Mexico. He says that he hopes he makes it across the border and sees Andy again, showing that he now believes in hope, contrary to his earlier remarks about how it was dangerous.