Should an art museum teach, and if so, how and what? Should it entertain? Is it a community center? These are the questions Charles Saumarez Smith explores in The Art Museum in Modern Times, a history of the spaces we devote to culture. Having run the National Portrait Gallery, the National Gallery, and the Royal Academy—a trifecta of glorious London museums—he infuses that expertise into his profiles, giving amateurs a primer in museology and professionals a fresh take on core issues in the field. Through origin stories, architecture, and local points of pride, he demonstrates the different answers modern humans have given to what an art museum is and should be.

The Altes Museum in Berlin, the Glyptothek in Munich, the Metropolitan Museum of Art in New York, and many other 19th-century museums look like temples for a reason. They sanctified art and proposed it as the province of scholars. With grand staircases and fat columns, and rows of Greek and Roman busts, they seemed forbidding. Paris’s Louvre is no less imposing—it began, after all, as a military fortress.

The Museum of Modern Art in New York changed all this. Its 1937 building wasn’t a Greek temple. Visitors entered straight from the street, with no imposing staircase, and the lobby was small. It was in the very heart of Manhattan, not a district devoted to high culture, and people could pop in on a whim. Though the museum was more inviting, the art within wasn’t exactly populist; its big-ticket items were often avant-garde French. This was the opening shot in the war against the 19th-century museum.

* * *

This war took many forms and Saumarez Smith examines some wild experiments. Lina Bo Bardi, for example, designed Brazil’s São Paulo Museum of Art in 1968, an enormous, low box. Inside, pictures were displayed on sheets of glass between steel rods, in the middle of big, open galleries, often stripped of their frames. This promoted a direct confrontation between visitors and the art. “My intention was to destroy the aura that surrounds museums,” she said, but in so doing she created the look of a warehouse—extreme informality run amok.

Then there is the Centre Pompidou in Paris, which opened in 1977. Saumarez Smith calls it a “Fun Palace” and it’s easy to see why. Conceived in the late 1960s, it’s not only a museum but a library, cinema, and public piazza suitable for outdoor sculpture and protests. The Centre Pompidou was, says Saumarez Smith, a “place where anything goes.”

The Broad in Los Angeles pushes the point further. It’s a fantastic building, with a Space Age look, reflective of its late patron Eli Broad’s Hollywood-sized ego. Broad revived the classic grand staircase—this time the grand elevator—that makes entering the place an experience. Once inside, visitors find themselves surrounded by his collection of contemporary classics, all very good Andy Warhols, Edward Ruschas, and Cindy Shermans, plus a Yayoi Kusama “Infinity Mirrored Room.” And the stunning exterior is covered by a white, pierced fiberglass and concrete skin called “the veil” that allows natural light into the building. Broad wanted a high-octane, glam presence and got one.

* * *

The collector-built museum, another of Saumarez Smith’s themes, tends often to become more about self-promotion than art promotion. Collectors like Charles Saatchi are strong-willed and passionate but also idiosyncratic and wildly experimental. The Guardian called him “a perpetual gorger of the briefly new, a junkie for shift and change and forward propulsion.” No surprise there. Saatchi was a P.R. man; his top concern was the taste of the moment. He chose an old factory in London for its big, open spaces and exposed pipes, but he painted it white to resemble a New York cutting-edge art dealership. The Saatchi Gallery, though it contains art, showcases Saatchi.

Eschewing self-aggrandizement is the Menil Collection in Houston, in my opinion America’s most exquisite museum. Dominique de Menil envisioned the space, opened to the public in 1987, as an anti-monument to her late husband, designed to capture the intensity of viewing art. She cared less about people learning from the experience—there are no labels—and more about the quality of the experience. What’s more, there’s no chronological arrangement of art, no audio tour, no special loan shows, no gift shop, and no restaurant. In a sublime Renzo Piano building, visitors look with no such distractions, in perfectly proportioned galleries, at the great art the Menils collected. The collection rotates to avoid a single piece becoming a fixture. It’s an exceptional tribute to the Menils’ aesthetic and vision.

The Barnes in Philadelphia is another early example of the collector-made museum, this one with an unfortunate twist. Dr. Albert C. Barnes, who made a fortune in antiseptics, assembled a massive collection of great works by Matisse and Cézanne, dozens of late Renoirs, and lots of bric-a-brac. He didn’t want the look of his museum changed one iota and he wrote his will accordingly. This set in motion a conflict between Barnes and Philadelphia’s conservative art elite who derided his taste. They hated him and he died hating them right back. In the late 1990s, the museum almost died, too. It was nearing insolvency due to Barnes’s restriction-heavy will, specifically his ham-handed investment mandate: low-interest bonds only. As a result, Philadelphia’s grandees persuaded courts to bust Barnes’s will, establish new governance, ditch most of his restrictions, and move the art to the center of the city. The new building is identical to the old (even down to its orientation to the sun) but it doesn’t work. In the original venue, the lighting was bad and hid much. Modern lighting shows Barnes’s gallery design suffers from sameness, and is often hokey.

* * *

Saumarez Smith also considers the phenomenon of the visionary architect—when designing an art museum unduly becomes not about the presentation of the art, but about the architect himself. For example, Frank Lloyd Wright added fireworks to the founding of the Guggenheim in New York through his inverted ziggurat design. A daring new look for Fifth Avenue, it wasn’t so much the appearance that roiled some, but his conception of the building’s purpose. Curators wanted a museum that accommodated “a business of trading, collecting, circulating, exchange and purchase, shipping in and out,” as Wright described it. That’s an office building, “and Mr. Guggenheim left no money to build one.” Instead, Wright designed a place to show art and provide visitors with a memorable experience, back of house be damned. The public might love his building but director James Sweeney grew to hate Wright, and generations of curators find his space impractical. On the other end of the spectrum, Carlo Scarpa, in renovating the Castelvecchio Museum in Verona in the late 1950s, made the castle a mere backdrop to the art, as opposed to a dominating partner. He installed wood and polished concrete floors, as well as rough plaster walls, neutering the castle architecture and giving the art independence. The visionary architect is often the impetus to radical change in museum philosophy, but he can also cause an array of problems.

Saumarez Smith finds a big shift around 2000. He compares the looming, aggressive hulk that is Marcel Breuer’s 1966 iteration of the Whitney Museum of American Art in New York with Renzo Piano’s new Whitney, moved downtown and opened in 2015. Breuer’s Whitney was a fortress with an identity distinct from its tony Madison Avenue neighborhood. Its façade, in fact, has only one window. The new Whitney overlooks the High Line and embraces its neighborhood of small commercial buildings, shops, and waterfront through hundreds of windows and nice views.

* * *

This leads to the new Los Angeles County Museum of Art (LACMA), now under construction. Swiss architect Peter Zumthor is the designer and Michael Govan the director. Zumthor’s only other museum is called the Kolumba, a small one in Cologne consisting mostly of religious art and medieval ruins. Critics call it a “museum of reflection.” Regarding his museum philosophy, Zumthor says it’s not about the architect, the city’s marketing, or blockbuster art. It’s about “the inner value of art.”

The new LACMA will be much smaller than the old. It will be a sleek and aesthetically reserved building—only two stories—with a serpentine footprint that sweeps across Wilshire Boulevard. Its interior will feature muted, graceful wood, metal, and stone. It will be the antithesis of all that’s flashy in Hollywood. But more to the point, the new LACMA’s size and gallery design make an encyclopedic, departmentalized display impossible. Govan envisions interdisciplinary, cross-cultural, and thematic displays. He wants it to be subjective rather than objective, “less of a monograph than a short story or poem.”

Govan is an acolyte of Thomas Krens, who directed the Guggenheim for years and pioneered the concept of affiliate Guggenheims, each with a distinct focus, all over the world. Govan hopes to do the same, building little LACMAs in neighborhoods throughout L.A. County. Each iteration will feel local, a point of neighborhood pride. I like this idea. A big, central museum like the Museum of Fine Arts in Boston, the Met in New York, or the Art Institute in Chicago feels Old World. It doesn’t make sense for far-flung Los Angeles.

The Art Museum in Modern Times is handsomely designed. It has a to-the-point look that fits Charles Saumarez Smith’s style and his studied detachment. It’s not art history. It’s an exploration of the ways the public and art can thrive together. That’s much more relevant.