By default, Joomla is pretty permissive with its usernames, allowing almost any character. However, for some reason, you don't want users on your website to use these characters. This plugin allows you to do this.
The plugin allows you to specify groups of characters that are not allowed in usernames and also min/max lengths for them. You can also apply these settings to the Name field.
This plugin works well on Joomla 3 and 4. After downloading and installing it, go to Plugin Manager > User - JMP Restrict Username to get started: