This is another problem from CTFLearn. In this problem, we were given 3 hints, that suggest us to find out about robots.txt. So I decided to just look at's robots.txt. And found this:

User-agent: *
Disallow: /70r3hnanldfspufdsoifnlds.html

So we can see that URL is disallowed, so maybe we should just check that out?
And when we visit we can see that it has the flag.


Link to the problem