As it comes from the Greek for harsh, rough, and bitter, I can see how it could apply to poverty.
Here is a nice write-up from dictionary.com:
4. Austere, bleak, spartan, stark all suggest lack of ornament or adornment and of a feeling of comfort or warmth. Austere usually implies a purposeful avoidance of luxury or ease: simple, stripped-down, austere surroundings. Bleak adds a sense of forbidding coldness, hopelessness, depression: a bleak, dreary, windswept plain. Spartan somewhat more forceful than austere, implies stern discipline and rigorous, even harsh, avoidance of all that is not strictly functional: a life of Spartan simplicity. Stark shares with bleak a sense of grimness and desolation: the stark cliff face.