An important point needs to be made too – it’s totally possible to have luxury items leshem shomayim in another way. Chazal say fhat a nice home, a beautiful wife, and good foos is “marchivin daato shel Adam,” it expands the mental abilities of a person. It can make a person learn better if their gashmius is taken care of.
But that has to be the intent, and if pursuing riches makes you learn less in practice, or you’re seeing that after that nice juicy steak, you want to lay down instead of run to learn gemara… it’s ear where your intentions are.