There is NO cheap accommodation. I stayed in Confort Hotel, the cheapest I could find, and it was 40,000 for a tiny room.
Also when you leave you have to pay 3,600 and there's no bank at the airport, meaning if you don't take your money with you you'll have to leave and go to the centre again to come back and pay it. Pain in the ass.
Not the best city in Colombia by a long stretch, but kind of necessary to get to the stunning La Guajira.
There's another place called castillo del Mar, located 15 min away from the center of town. It's not great but it's the cheapest in town at around 25k. If you are on low budget that will do. And you can get some info on tours to Alta Guajira. The staff is not that nice but if you get to meet the old German owner of the place, he'll tell you some interesting stories.