People called out Tears of the Kingdom for this specifically (the DLC/expansion pack meme that is popular) and I've seen some people call out ER, myself included, for boss recycling, overworld encounters (stage coach robberies), side dungeon tile sets, and itemization in terms of reused content. The difference in those games is that they can hang their hats on other strengths. ER has best in class combat and boss designs. Zelda has best in class traversal, physics, and sandbox elements. BGS games usually fell really dated in a lot of areas when it comes to tech and gameplay systems. What they do well is exploration, character customization to fit a play style, and item persistence. Those first two items being the most important. So if Starfield's exploration element is perceived to be lacking, that's a huge missing pillar that is going to effect user/professional reviews. Someone mentioned earlier about no other gaming letting you do the laundry list of features that Starfield offers but my counter would be, "which of the elements is best in class?".
We also need to consider series fatigue and expectation of standards. Yes, Starfield is a new IP but it has roots in TES and Fallout. Some people will never experience this fatigue but others will love an entry in a series with the caveat of "...but I wish it did this." And when the next entry comes along and the "...but I wish it did this" is still a problem, they'll be less forgiving. And with BGS games having so many shortcomings despite their development time and budgets, it's hard to fault anyone for expecting better gameplay systems, better animations, more competent technical experiences, etc. My breaking point was Oblivion with Morrowind being one of my favorite games of all time. Time will tell if Starfield is that breaking point for others.