(If not, is there a way to make it so?)
In pure MySQL land, this will depend on the underlying character set and collation of the underlying database and tables that Craft is searching against. You can read more on the MySQL 5.1 rules regarding that here and see an example of how to utilize it here.
In Craft, however, we will do some normalization and additionally swap any known diacritics with their ASCII counterparts (ü => u) among other things before saving to the
craft_searchindex table in the database.