hckrnws

OP here.

site: https://knock-knock.net

Every server with port 22 open gets hammered by bots trying to brute-force SSH. I built a honeypot that accepts every connection, records the credentials they try, and displays it all on a live dashboard with a 3D globe.

Some fun things you'll notice:

- Bots try the same passwords everywhere — "admin", "123456", "password" are the classics. Yes, you'll see the Spaceballs password in the top 10.

- Certain countries and ISPs dominate the leaderboards

- Attacks come in waves — sometimes nothing for a minute, then a burst of 50 from one IP cycling through a wordlist

- There's a knock-knock joke panel because I couldn't resist

Originally inspired by my kids asking "who keeps trying to log into your computer?" when they saw me tailing SSH logs.

The stack is Python (FastAPI + paramiko for the honeypot), Redis pub/sub for real-time updates, SQLite for stats, and globe.gl for the visualization. WebSocket pushes every knock to your browser as it happens.

The whole thing runs on a $6.75/year VPS. The domain costs more than the server.

Source: https://github.com/djkurlander/knock-knock

Beautiful. Have you considered adding a "replay certain timeline" feature so that users get the feel of the throughput and emergence much like Gource [1] did for git?

[1] https://gource.io/

Hadn’t considered it, but that’s a nice idea. All of the necessary info, with time stamps, is already recorded in a SQL database, so it wouldn’t be difficult to replay events.

This is neat. What VPS service do you use? I am trying to replace my tendency to spin up small EC2 instances just to deploy a simple web app.

My $6.75 per year VPS was a Black Friday sale from Dedirock on https://lowendtalk.com. Some of the Black Friday sales are still being honored. The site https://cheapvpsbox.com/ has a nice search engine for cheap VPS sales.

Note: just be sure to have some sort of backup solution because when a deal seems to be too good to be true, sometimes the company will go under.

I had that happen years ago, consequently it meant my first ever VPS disappearing.

I think the deal back then was like 15 EUR per year.

Scaleway has small instances (Stardust) btw: https://www.scaleway.com/en/pricing/virtual-instances/

They seem expensive otherwise so I’d go with Hetzner for most other stuff. Heck I’ve even used Contabo too (they don’t have the best reputation, but it worked out okay for me).

I recommend a dedicated $40 hetzner or OVH box and just keep all your projects on that. They're pretty powerful. I was spending a lot on a bunch of $5 linodes until recently and you have to keep them upgraded etc...

how deep are your WebApps? Cloudflare pages and workers have a generous free tier, depending on what you're doing.

Do you have any insight on SSH servers that only allow login with public key authentication? Do bots leave immediately when they see that they can't use passwords?

If the bot sees no login / password sequence, there’s no way for it to brute force credentials. If the server only takes ssh keys, that will cause an immediate disconnect. Which is why this setting is best practice when setting up a server when practical: PasswordAuthentication no.

I wish this would be the default. I expose my homelab port 22 directly to the internet. I'm _pretty_ sure I always always always disable password auth but I do worry about it because most distros have an unsafe default.

(A lot of this risk is mitigated by not having login passwords but I definitely have one node where I have a login password, it's an old laptop so I thought I might want to physically log in for local debugging).

I guess the ideal solution here is to run a prober service that attempts logins and alerts if it gets any responses that smell password auth is possible. But no way I have time to set that up.

You can do this in about 5 lines of bash with a cron job. Something like:

  ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no -o BatchMode=yes user@host 2>&1 | grep -q "Permission denied" && echo "password auth enabled on host" | mail -s "SSH audit alert" you@email.com

The tricky part is that sshd_config can be overridden per-user with Match blocks, so ideally you'd probe with a few different usernames. But even a basic probe catches the 90% case of someone forgetting PasswordAuthentication no.

For the laptop with a real login password: you could set PasswordAuthentication no in sshd_config but keep the login password for local console access. Those are independent settings - sshd_config only affects remote SSH, not local login.

When you get a "Permission denied (publickey)." if you try to connect to a server which requires a public key for authentication, it causes your 5 lines to wrongly raise an alarm ... you need to adapt your grep.

One way to solve this it to use a configuration management tool (Puppet / Chef / Salt / Ansible etc.). Alternatively, run NixOS. You apply the setting once and then it's applied to all your machines from that point onwards.

I do run NixOS, but it's easy to make mistakes in a complex setup.

Cool project

But also wanted to let you know about

https://objective-see.org/products/knockknock.html

And knockd: https://wiki.archlinux.org/title/Port_knocking

Common name in case you wanted to differentiate yourself a bit

I was aware of port knocking, but not the Mac malware scanner with the similar name. Good to know!

You probably also know of Netbird -- open-source zero-trust VPN.

Personally, I shall some day find the patience to code and test a poor man's zero-trust -- app/site knocking + firewall whitelist.

Very nice! I am looking forward to many people running this. Perhaps people could add their URL in a ./contrib directory or something to that effect? I might set this up when I get back from the feed store.

Nice idea. The original VPS is in Los Angeles, but I installed the app more recently on VPS's in London, Tokyo, and Amsterdam. I've been noticing some interesting regional differences, but it may just be smaller sample of knocks for those sites so far. I'll set up that contrib directory so that we can share our dashboards. I would be interested in looking at others' dashboards to suss out patterns.

Side question: which cheap VPS are you using in Los Angeles? Looking to get one in the Southern California area.

My $6.75 per year vps was a Dedirock Black Friday sale that I found https://lowendtalk.com. https://cheapvpsbox.com/ reports several nice Los Angeles sales still going on from various providers. My London, Tokyo, and Amsterdam VPSs are holiday sales from RareCloud and Racknerd - all less than $19/year.

Before I saw this comment I was curious and used dig+ARIN to look up the IPs and saw they were at Cloudflare. Given how rapidly the data changes and that the updates are via Websockets, do you get benefits from them serving assets, or is that to obscure the origin so it doesn't get extra attention, skewing the results? Cool project!

Good observation. I am using a Cloudflare orange cloud proxy to hide the IP address. I’m also blocking direct access to my web server by IP addresses to make it that much more difficult to associate the IP address with my domain. Most people installing knock-knock probably won’t care, but I figured that this would be worthwhile for the “official” server. Instructions for setting this up are in the extras/ufw-cloudflare directory of the repo. Yes, there are other ways to track down the IP address, but they are a lot harder.

By the way, I noticed that the bots were guessing usernames like “knock-knock” before blocking direct IP access to the web site. Looking at the other passwords guessed, I realized they were extracting words from the title of the index.html! So it’s all about masking the server’s identity - I’m not really getting other benefits out of Cloudflare.

contrib directory added!

> who keeps trying to log into your computer?

I'm curious, how do you think this helps you answer the question? Proxies are incredibly easy to come by these days, rotation makes it hard to identify what's behind it all.

That’s a valid point. We can easily see where the attack is coming from but not who or which botnet. Some of these can be inferred by the pattern of usernames and passwords attempted, and the ISPs. Someone suggested that I collect the client SSH signature as well, which would help. But you’re right, we don’t know who is behind the attacks.

I'm guessing the SSH signatures can rotate as well. I remember someone did an analysis of rotation patterns for HTTPS requests; that's when they saw some interesting clusters.

I saw an ISP called Microsoft, USA… is that an official microsoft computer doing that or something else?

Yes, Microsoft shows up a lot. Some of these bots are running on Azure.

My favorite ISP to spot occasionally is SpaceX / Starlink. That can’t be the most economical ISP for bot traffic, but machines can be infected, even on Starlink.

Starlink bot here, but you won't see me because I'm behind a VPN

Awesome, I loved it thanks for sharing it.

And I remember more than a decade ago I went down the rabbit hole hunting these bots and indeed, I found Netherlands was always the king of hill when it comes to bots, followed by US, Netherlands still there I see.

Some things never change.

One of my favorite visualizations for this is to switch to the globe view and choose the “HEAT” style for a 3D heatmap superimposed on the globe. Green means few hits, and red signifies lots of hits. The Netherlands is so small that it’s tough to see though!

Wow that's fucking beautiful, man. That's beautiful. Wow, I love that!

What $6.75/year VPS do you have?

Was gonna ask the same question. nearlyfreespeech perhaps? They're quite cheap. Haven't seen any other providers at a similar price point.

They answer it down in the thread I found. https://cheapvpsbox.com/

Well done, OP.

It has been 24 hours since this post went up, so here are some fun stats.

During this time, there were 13,024 knocks on the server from 368 unique IPs. That's ~35 knocks per bot.

During this time, thanks to Hacker News, there were 23,556 visits to knock-knock.net by 15,946 humans. That's ~1.5 visits per human.

So in the last 24 hours, we actually had nearly twice the number of human visits than bot visits, and 368 bots put on a show for 16,000 humans!

Cool site, I think the fact that bots WILL try to get access to your server as soon as it's publicly available should be mentioned more in basic tutorials etc... I remember panicing when I had set up my first webserver as a teenager and was checking the nginx logs out of curiosity, I thought this was a real threat to the security of the server and almost shut it down lol.

This is great.

Do you publish a list of the 'knocking' IP addresses anywhere? (abuseipdb.com was mentioned, maybe I need to just pay for their service for their 100k blocklist)

(I've mentioned this before on related HN threads) I've got a setup whereby any incoming connections to ports behind which I don't have a service running get logged, and periodically the log is filtered and the IP addresses extracted and added to a block list.

My theory is that, if there's traffic coming into a port behind which there's no service (and therefore there's absolutely no good reason for this traffic to exist), then it must be malicious. If it's malicious, then I have no reason to trust any data coming from that IP address.

This is based on OPNSense firewall rules and logs and is documented haphazardly here: https://github.com/UninvitedActivity/UninvitedActivity

Most IP addresses age out of the logs after 12 months. I also have lists of common internet scanners that I've got from my own curation of the logs plus other similar projects of others. I'm just protecting my little homelab, so I don't care whether I'm blocking an infected computers, computers running proxies, or blocking large swathes of the internet via ASN blocks. What I have setup is a pickaxe, where a lot of people really need a scalpel. Don't apply blindly!

(But I do think that if there was more aggressive blocking of the malicious traffic on the internet, then there would be more motivation for providers to at least attempt to minimise facilitating it - I admit that there is a fine line, and opinions on what is and is not malicious are subjective)

I'm reporting the bots that have visited to abuseipdb once per day, but yeah, there should be a free alternative. You aren’t the first person to have asked for this.

It would be trivial to write out a file that people can grab for free. What do you think would make the most sense? Plain text file, one ip per line, of offending ip’s within the last month? Or year? Or a .csv with the dates included? Generally I’m a big fan of simplicity.

Plain text file one IP address per line works for me. Simplicity for the win.

Within the last month is probably enough. If I was consuming it, I'd add each monthly list to a database so I can build up my own 12-month (or whatever time frame suits me) list over time.

Or, publish one list for the last month and one list for the last 12 months.

Keep up the great work!

OK - thanks for the excellent suggestion. It's now implemented (just two SQL queries that will run as a cron job every night). You can grab the month and year offending ip blacklists this way.

wget https://knock-knock.net/static/ip-blocklist-month.txt

wget https://knock-knock.net/static/ip-blocklist-year.txt

Great visualization!

I currently accept and then close/drop the connection "unclean" (no FIN or RST packet). I do this in hopes that the offender will waste some resources (time) thinking it is still connected while I spend minimal resources.

My reasoning is that if enough servers implement such measures it will become very costly for the offenders to scan.

Perhaps I can also add some logging to build a IP blacklist as described below.

Nice, are you using something like this?

    iptables -I OUTPUT -p tcp --sport 22 --tcp-flags RST RST -j DROP
    iptables -I OUTPUT -p tcp --sport 22 --tcp-flags FIN FIN -j DROP

No I think I wrote something in C (it was written a while ago) accepting the and then discarding the connection in such a way the RST/FIN was never send, making sure to clean the socket server side.

I guess a timeout will need to be adjusted/implemented on the bot's end I remember fixing a similar bug at work and it was quite involved. At any rate the very least the connection was made and discarded.

I guess the iptables solution would also work well and you would have a correctly working serverside.

T-Pot does something similar: https://github.com/telekom-security/tpotce

Thanks for pointing that out. T-Pot is cool and a more general honeypot framework. Potentially I could have built knock-knock.net on top of T-Pot.

Personally, I just move SSH to a random high port on each of my public servers. Works like magic, no more log noise.

This is very interesting to me, would most of these bots be running on servers that have already been compromised? If that's the case, is the Netherlands/Digital Ocean the most common combo as it's what most normal people use, or is there some other reason bots favour it?

Many/most of these are servers that have been compromised. DigitalOcean is certainly one of the biggest ISPs/providers; however, I’m betting that if you looked at ratio of knocks per ASN IPs registered, DigitalOcean would still be at the top. I’ll look into that.

Providers can shut down abusive IPs. I run a script every night to report attacks to abuseIPDB.com (included in the extras folder on the knock-knock GitHub repository). Some providers just don’t care.

> Some providers just don’t care.

And they should be shunned by everyone. We should all be naming and shaming such providers and those of us with any conscience at all will avoid using them. This is the only way to stop the tsunami of bad actors.

Or you could just choose a strong password / disable password auth? Why would DO side with anonymous abuse reporters over their customers?

Though strong passwords or preferably ssh keys are important, there will always be servers with weak passwords.

And DO doesn’t have to side with individual abuse reporters. If they cared, they could spend a fraction of an hour setting up the knock-knock software on one of their own servers, and generate their own list of abusive IPs. They just don’t care.

Is this hosted on DigitalOcean, say in The Netherlands? Could it be that spam traffic within the same datacenter bypasses their detection?

No, knock-knock.net is not hosted on DigitalOcean, and all 4 of my other knock-knock servers, using different providers, and distributed geographically currently have DigitalOcean as the worst offending provider.

>user: claude password: claude2026!

>user: claude password: claude123

I wonder if these have come from leaks or if someone has a script that generates the top ~xx most likely passwords based off the username.

Some of the passwords definitely come from leaks, but adding 123 or 2026! to the end of a frequent username is a surprisingly common pattern. Lots of suffix variants: 123!, 2025, 2025!, @2025, @123, etc. In fact, the Trivia pane of knock-knock.net points out when the password is just the username plus a suffix.

This is cool but doesn't Grafana + Prometheus have a lot of this included already?

My understanding is that they are a more general purpose data collection, and visualization framework. Potentially you could build something like this with that software, but they do not have knock-knock.net’s functionality built in.

I wanted to ssh hello-hacker-news@knock-knock.net, but sadly it doesn't work because the site is hosted behind Cloudflare. You'd have to work out the true server IP address which is not easy to do.

Sadly? Intentionally! The IP is hiding behind Cloudflare mainly to make it much harder for the bots to figure it out. Blocking you from messing with the stats is just icing on the cake. :-)

I don't think hosting the site behind Cloudflare will affect the number of SSH brute-force attempts, these bots are just brute-forcing the entire IPv4 space aren't they?

Exactly right. I just didn’t want the bad actors to know my own IPv4 address so that they might try to cause havoc or treat my site differently.

Roger.

And cool project btw.

If you are using fail2ban, is the amount of hits been reduced? Or you can see them anyway in the logs?

Fail2ban would cut down on the noise quite a bit. I’ve installed it on other servers and have recommended it to others. But then we wouldn’t have all of this beautiful bot traffic to visualize.

Very interesting that DigitalOcean is by far the largest source.

Other (more responsible) VPS providers, e.g. Linode, actively block machines from which they detect a lot of abuse traffic. Wonder why DO doesn't do the same.

Hetzner would have blocked some of those pretty quickly. Especially the top 2-3 IPs that seem to have thousands of attempts.

Very fun site. Cool idea indeed. I think it's a neat piece of art. I wish I could scroll sideways, though. The page got cut off for me.

If you are on the desktop, you should be able to scroll sideways either by choosing a menu icon at the top, or by clicking on a panel (which will rotate the panels to the left). On the phone you can visit a panel by choosing the icon from the rotating carousel at the bottom, or by swiping the panels to the left or right.

This is fascinating! Did the attempts start as soon as you started up the server, or did it take a little while to get going?

The attempts started almost immediately, but they certainly accelerated over the course of the first few days. Initially, I think it was less than 1 knock per minute, but now it’s over 8. As a comparison, I set up https://amsterdam.knock-knock.net yesterday, and it is now at 2.9 knocks per minute.

The bots though scan through all the IPs on the internet, but perhaps they bias certain IPs (local / faster response? On the bots provider network?). Will be interesting to watch this over time.

Seems like DO sure has a bot problem. I wonder what percentage of their business is less-scrupulous actors.

Something I've thought about is how does a VPS provider prevent this kind of thing?

Most of this kind of traffic goes by completely unknown and therefore unreported, so 'VPS host X' has no case to answer, to some degree.

If malicious traffic gets reported and 'VPS Host X' takes action and either contacts the operator of the VPS or shuts down the VPS following a traffic investigation, then the operator of the VPS creates another one on 'VPS Host X' or 'VPS Host Y'.

(all questions are rhetorical, not directed at parent) Should VPS Hosts, by policy, block outgoing connections to port 22? Where is the line drawn for default blocking policies? Block everything and force the operator to configure a firewall to specify which ports the VPS can connect outwards to (or all ports)? At some point there will be friction that discourages customers and affects sales / profits, and therefore a disincentive to try to clean things up.

Secondary effects, more aggressive blocking of malicious traffic could potentially allow for some/more/better reputational differentiation between VPS hosts to offset loss of customers due to better security friction.

I doubt there's any legislation coming anytime soon to enforce a certain level of internet hygiene.

There is no such thing as a "good reputation" datacenter ip. They should all get blocked by anyone who cares about bots.

You're assuming the owner rented the VPS to run the but but it's more likely intended for something else and is infected with malware / some intern being cute. After all there are cheaper plans than DO.

> it's more likely intended for something else and is infected with malware / some intern being cute

Nah, DO offers free credits so threat actors just keep abusing that, it's really easy to make (or buy) tons of fresh trial accounts.

Ah, that makes sense. I’ve been wondering why DigitalOcean has so much of the bot traffic.

This is great, now I can greatly refine my dictionary for trying to brute force knock-knock.net.

!!!

Good luck trying to log in via port 22. The real ssh port is located elsewhere and doesn't accept passwords. :-)

…but if it did accept a password it would be 12345.

Now I want to see one where you let any login work but dump them at a fake shell that logs the commands sent. I’m curious what they do. Could even crowd source a mapping of command string matches to example output.

This is great. I run a server for my blog and can confirm idiotic bots continually hammer port 22. Sometimes I check my SSH logs just to see what is going on but I’ve never detected anything cleverer than trying common username/pw combinations.

It seems a little pointless, surely every server actually accepting SSH passwords has been 0wned year ago.

Even on a random port (well I picked port ___22) I get random SSH attempts.

My solution is convoluted: On my NAS I have a PHP form that accepts a password, when it's correct, set a flag (in the form of touching a file), and every minute a cronjob runs a bash script to check for the existence of the file: if it exists, then run a python script to talk UPnP to my home router to tell it to forward port ___22 to my NAS' port 22.

Hmm, probably running a VPN server, like WireGuard, makes more sense..

I have gotten what looks like SSH, TLS, HTTP, and other things, on various ports.

Another possible way would be port knocking. (I had previously set up port knocking on my HTTP server, but there seems to be a bug in the kernel (or in some driver) that prevents it from working correctly, so now the HTTP is not available. Using port knocking to restrict access to HTTP is probably not common, and might prevent your solution from being used if the form uses HTTP.)

I just disable SSH passwords and force using a certificate, which should be immune to bots barring some horrible unknown flaw in the ssh daemon.

Running over a VPN service would have the much the same effect.

I know, at some level, it seems crazy that the bots are spending so much time on this. However, there are plenty of machines on the Internet, and presumably most of these bots' machines were captured using this same technique.