Born in Russia, Weber emigrated with his family to New York as a child. His earliest artist training was with Arthur Wesley Dow, who encouraged his students to reject traditional narrative painting in favor of new explorations of expression and form. Weber was one of the first American artists to incorporate "primitive" influences into his work. He also studied in Paris at the Academie Julian, where he learned from contemporary Fauve and Cubist painters (even took classes with Henri Matisse) and became friends with Henri Rousseau. Weber would be responsible for Rousseau's first exhibition in America and he also helped to introduce Cubism to an American audience after his return to New York in 1909. Friends with many experimental artists in Paris, Weber was responsible for sending the first of Picasso's paintings to America for exhibition.

Despite his early success in France, Weber often felt that his work was unappreciated and his prickly response alienated him from many colleagues and potential supporters. Although he was once a central member of the Stieglitz circle, even living at Stieglitz's 291 Gallery when he was very poor, the men had a falling-out in the early 1910s; he was excluded from the Armory Show when he protested the small number of works he was invited to submit. In his later years, Weber turned to more representational, often expressionistic renderings of Jewish life and culture.