You could probably do this with some smart machine learning on the actual glyphs representing the Unicode codepoints. You will need to build the mapping of Unicode characters into latin characters which they resemble. Since the encoding that turns 'the Family' into 't���� ��������������y' is effectively random and not following any algorithm that can be explained by the information of the Unicode codepoints involved, there's no general way to solve this algorithmically. Creating arrays and replacing manually? Or does this language have native functions about this issue? However, we can make the conversion 'òé����öç->oeisoc' in a simple way. For instance, I said 't���� ��������������y -> the Family' conversion in the example but it seems difficult convert it completely. You know some people use the diacritics as funny characters and I have to remove that as much as I can do. Actually I'm not working with arabic languages or something like that. The lookup arrays (as it was in our case) took perhaps 1 man day to produce, to cover all diacritic marks for all Western European languages.
This comes from experience of having worked on an application that was required to allow end users to search bibliographic data that included diacritic characters. If your language supports native Unicode characters (as Java does) and optimises static structures correctly, such find and replaces tend to be blindingly quick. It then becomes a small amount of slightly boring work creating the dictionaries, and a trivial task to perform the replacement. Similarly, the Unicode character set contains hundreds of mathematical and pictorial symbols: there is no (easy) way for users to directly enter these, so you can assume they can be ignored.īy taking these logical steps you can reduce the number of possible characters to parse to the point where a dictionary based lookup / replace operation is feasible. Consider the source of the input - if you are coding an application for 'the Western world' (to use as good a phrase as any), it would be unlikely that you would ever need to parse Arabic characters. If you must, for whatever reason, convert characters, then the only sensible way to approach this it to firstly reduce the scope of the task at hand.
This is before you even go onto consider the Cyrillic languages and other script based texts such as Arabic, which simply cannot be 'converted' to English. As others have pointed out, diacritics are there for a reason: they are essentially unique letters in the alphabet of that language with their own meaning / sound etc.: removing those marks is just the same as replacing random letters in an English word.
ASCII CODES FOR SYMBOLS CSS TRICKS CODE
David you know some EMOs use different chars in sentences.2 Why do you want to do this? If we knew what your overall goal was, we might be able to be more helpful.1 Should your third example be �� ��� Y?.See this question: /questions/249087/… - there should also be some other questions about this topic, but I can't find them at the moment.How can I convert all these with Java? Please help me :( Just try scrolling down and see the variations of letters. The complete list of unicode chars is at or. Īnd I saw that there are more than 20 versions of letter A/a. The problem is that, as you know, there are thousands of characters in the Unicode chart and I want to convert all the similar characters to the letters which are in English alphabet.įor instance here are a few conversions: ��->H ��->V ��->Y ��->O ��->C t���� ��������������y -> the Family. Convert Accented Characters Quickly in Excel
0 kommentar(er)
