The awkwardness fades with a set plan and reps, no one is watching you, they are focused on themselves.
The quick answer
Almost everyone feels this at first, and the honest truth is other people are absorbed in their own sets, not judging yours. The fastest cure is walking in with a written plan so you never wander unsure of what to do next. Learn a handful of machines and free weights, go at quieter hours at first if it helps, and give it a few weeks, familiarity kills the anxiety.
How to think about it
No one is watching
The spotlight effect makes us overestimate how much others notice us. In reality gym goers are counting their own reps, resting, or on their phones. Even experienced lifters were beginners once and mostly remember their own nerves, not yours.
Walk in with a plan
Most awkwardness comes from standing around unsure what to do. Have your session written out so you move set to set with purpose. Knowing your next exercise removes the aimless wandering that makes people feel exposed.
Build familiarity
Comfort comes from reps in the room. Start with a few machines you understand, add exercises as you learn them, and consider off peak hours early on. Within a few weeks the place feels routine and the self consciousness fades.
Do this
Bring a written planLog your session in advance so you always know the next exercise and never wander.
Learn a few staplesMaster a handful of machines and lifts first so you feel competent, then expand.
Try off peak hoursGo when it is quieter at first if crowds spike your nerves, then ease into busier times.
Give it a few weeksKeep showing up, familiarity with the room and routine is what dissolves the awkwardness.