Part 1 Part 3

This week, I learned the hard way how a dependency can break your whole app.

My music app, InstaPlayer depends entirely on the SpotiFLAC engine to actually fetch songs. In the latest release, the author put a Cloudflare Turnstile captcha in front of the download requests. That means my backend can no longer just ask for a song and get one back, but it must involve a real human behind the scenes to solve the captcha. This absolutely demolishes the user experience, my download flow, and everything I had been planning around it.

flowchart LR
    B@{ icon: "logos:docker-icon", label: "InstaPlayer backend", form: "square" }
    T@{ icon: "logos:cloudflare-icon", label: "Cloudflare Turnstile", form: "square" }
    B -->|download request| T
    T -->|looks like human| S[(Music provider)]
    T -->|"looks like bot </br>(missing cookies, sus IP,...)"| X[Blocked]

Discovering nodriver

I love web scraping, and if you do too then you know captchas are the biggest enemy. Previously I just ignored them to go a different route but this time, it’s not possible and I had to look for some direct soltuion. I found nodriver, a python package built to get through exactly this kind of Turnstile captcha. After a lot of effort I got it working on my own machine, and for a brief moment I thought that was the solution I needed and I could move on from this mess.

The IP problem

Then I deployed it to my server and it stopped working. The reason is that my server’s IP is blocked by Cloudflare, which is honestly a given. Servers live in datacenters, and it’s not surprising that Cloudflare doesn’t trust this source.

flowchart LR
    CFh@{ icon: "logos:cloudflare-icon", label: "Cloudflare", form: "square" }
    CFs@{ icon: "logos:cloudflare-icon", label: "Cloudflare", form: "square" }
    H[Laptop/Homelab] -->|residential IP| CFh
    CFh -->|looks human| OK[Allowed]
    S[Rented server] -->|datacenter IP| CFs
    CFs -->|looks bot| No[Blocked]

The homelab plan

I plan to build a homelab in a few months with my old laptop, which gives me a real IP address I can use for this. A home connection comes with a real residential IP, which hopefully gets Cloudflare’s trust. I don’t know exactly how well this will hold up, but it is the most plausible and cheapest option I have right now. I am a broke college student, so cheap matters a lot to me.

A possible shortcut

Today, something unexpected happened. Someone from anyip.com looked at my repository and suggested that their service could solve this for me by providing a real residential IP adress. That could open up a much nicer path where I do not have to sacrifice my old laptop. The downside is the pricing. I checked the website and five gigabytes of requests costs twenty five dollars, which is well out of my budget. I am going to reach out to their representative and see if there is any room on pricing for a small project like mine.

flowchart LR
    Need@{ icon: "logos:cloudflare-icon", label: "Needs legit-looking IP", form: "square" }
    Need --> Home[Homelab, residential IP]
    Need --> Proxy[Paid proxy, anyip.com]
    Home --> Fixed[Downloads work again]
    Proxy --> Fixed

What this could teach me

If that path works out, it changes the kind of problem I get to work on. Now, every request costs real money, so I would have to learn how to optimize request volume and cut out anything wasteful. It would be my first time working with a paid API where my actions have consequences, and I am quite excited about that.

Stay tuned to see how I chase free music!