Solving the Captcha Without a Homelab

Part 1 Part 2

Last time I wrote about Cloudflare Turnstile breaking InstaPlayer, and about my plan to fix it by turning an old laptop into a homelab so my downloads would come from a residential connection. I did get it working but with a proxy instead of self-hosting.

Since a recent SpotiFLAC update, the provider endpoints only answer requests signed with a key from a valid session, and you can only get that valid session after passing the Cloudflare Turnstile captcha. Sessions expire after about six hours. So my initial thought was to design a system that forwards that captcha to my phone and I would solve it every 6 hours. But that sounded exhausting.

I had everything figured out for a system to bypass that, except the captcha itself, which checks cookies, IP and all sorts of stuff to make sure a human is actually solving it. Luckily, I found a tool called nodriver designed specifically for this. I registered a handler that hands the challenge URL to a small solver that uses nodriver, which drives a real Chrome inside a Docker container.

flowchart LR
    E@{ icon: "logos:go", label: "SpotiFLAC engine", form: "square" }
    V@{ icon: "logos:python", label: "my solver", form: "square" }
    C@{ icon: "logos:chrome", label: "real Chrome in Docker", form: "square" }
    T@{ icon: "logos:cloudflare-icon", label: "Turnstile", form: "square" }
    E -->|challenge URL| V
    V -->|nodriver| C
    C -->|solves| T
    T -->|token| E
    E --> S[(session, valid 6h)]

Unbeknownst to me, finding nodriver was just the start.

Two captchas

I spent more time solving an unrelated problem than actually bypassing the Captcha. The problem was that due to some finicky default configs of nodriver, two Captcha frames are rendered on top of each other, preventing my code from locating the box to click on. And because I ran Chrome in a headless container, I could not use my eyes to see what the problem was. It ended taking me 2 days just to find out the box isn’t invisible.

My clicks were landing on nothing while the debug tool told me the coordinates were right.

I also learned about fingerprinting while making this Captcha bypass feature. Previously, I already knew that Captcha tools don’t actually check if you click the right images of “cows” or “cars” but they check if your browser is of a real user. But now, I learn the extent to which it checks a browser (and the fact that the check can be bypassed).

Turnstile checks for a lot of things. WebGL is the browser feature that draws 3D graphics using your graphics card, and a real computer always reports some renderer for it. The browser’s timezone also has to match its place of origin. The screen resolution has to look like an actual monitor instead of the odd default size when it’s launched in a container. The browser has to run on a display, even a fake one, rather than the headless mode. The mouse has to travel to the checkbox the way a hand would instead of teleporting onto it.And there shouldn’t by any trace of an automation tool. All of these had to be right at the same time, which is the part that took the longest for me.

Fake IP

After I faked the browser’s fingerprint, I managed to get it working on my local machine. There was only one thing left to take care of: IP. In the last part I did realize that making requests from a datacenter’s IP would alert Cloudflare, and I planned to solve that with my homelab. But I found a less exhausting approach, that is to pay for a residential proxy. For 5 dollars, I get 5GB of traffic which will last me the next 3 years. The reason is that I only route the Captcha and Token I obtain from the Captcha through that proxy. Song downloads are routed normally through the datacenter.

flowchart LR
    S@{ icon: "logos:docker-icon", label: "my server", form: "square" }
    P["proxy"]
    D["datacenter"]
    T@{ icon: "logos:cloudflare-icon", label: "Turnstile", form: "square" }
    M[(backend)]
    S -->|"captcha + token<br/>~75 KB"| P
    S -->|"songs<br/>30 to 50 MB"| D
    P --> T
    D --> M

What I learned

Looking back, this issue taught me how to really mask a browser against Captchas, which would probably come in handy in many places. I also learned how much slower debugging gets when you cannot see the screen. I was also proud of the fact that I optimized the proxy traffic to save me a lot of money.