During the 1910s and 1920s, the branch's collections included materials in more than 10 different languages, mostly in response to immigrant patrons from Eastern European and Mediterranean countries. At the time the branch offered citizenship classes. The library held a significant number of items in Yiddish, reflecting the large Jewish presence in the neighborhood until the 1960s. The branch was the home of the Yiddish collection until the 1960s.
During the 1930s and 1940s, the branch housed a Japanese collection. This collection served Japanese Americans from as far away as Bainbridge Island and Issaquah. (There was no King County Library System at the time.) The branch was held to be the best place for the Japanese collection because of its proximity to what used to be called Japan Town. Very shortly after the attack on Pearl Harbor, the collection was removed, never to be returned.
In 1965, the library system established the Negro Life and History Collection with 352 titles. In 1967 and 1968, the downtown library administration threatened to close the branch because of declining circulation, but the Black Friends of the Yesler Library and Alpha Kappa Alpha Sorority worked to develop the collection, which celebrated African American culture and history. As a result of these efforts, patronage increased.
By 1984, the branch housed one of the nation's largest collections (5600 titles and 25 newspapers) focusing on African American history and culture. In September 1988, the branch was renovated at a cost of $790,000 (original cost of construction in 1914 was $40,000).
