Limit that SSH as must as you can!!!!
1-You should install fail2ban somehow (never tried on Endian). Fail2ban is the defacto standard for blocking brute force attacks. It works on many services, not only SSH, but it's a bit hard to setup.
https://www.fail2ban.org2-Limit SSH access to some IP ranges, the ones you know you'll connect. Like for example your country, or your ISP/cellphone ISP. Also limit access to only the SSH port.
3-Change your SSH port, just to avoid scanners.
4-As an increased security measure, you can also try 2 factor authentication, SSH works fine with Google Authenticator/FreeOTP. So you'll need the password and a token (from your Android phone) to access it. Yet again, I never installed it on Endian.
I think this should be the optimal security you need to secure SSH properly.