Artificial intelligence and discrimination : a key issue

To say that computers are overtaking humans in many
ways is nowadays an understatement. Computers have far greater compute and
storage capacities than humans and solve problems that were previously
unsolvable. However, the reverse is also true: some tasks remain an element of
mystery for computers. One task, in particular, characterizes this phenomenon: language
comprehension. Although this is a skill that humans acquire early in life, it
is very complex to make a computer understand the meaning of words. Worse, it
is even more difficult to explain sentence construction, syntax, and especially
semantics This problem, which has remained unsolved for a long time, would
however have applications in many fields. Probably the most obvious are
translation, automatic text correction, self-completion of words or word
suggestions, which are now widely used in mobile phones. Some might also dream
of computers, writers, novelists, journalists. So how do you teach a computer
to understand languages? The key to being understood. MathIn order to make human language
understandable by a computer, it is necessary to translate it into what it can
understand. One of the main paradigms is to create a mathematical representation of the
sentence. The most classic representation is the so-called vectorial
representation: each sentence, document or word will be represented by a table
of numbers, i.e a vector, allowing its content to be summarised and represented
in space. The granularity of this representation, i.e. the amount of
information contained, will vary according to the complexity of the chosen
representation. Statistics are
part of the dance<o:p></o:p>

Historically, one of the first statistical representation
is the bag of words. The principle
is as follows: a sentence is represented by a vector made up of 0 and 1.
This vector is the total size of what is called the vocabulary, the set of
words that the sentence could potentially contain. Each word of the vocabulary
is associated with a box in the vector. If the word is indeed present in the
sentence, then the number in this box will be 1. If the word is not present in
the sentence, the number representing it in this vector will be 0. Thus, the
word bag mechanism allows a sentence to be represented by the presence or
absence within it of each word in the vocabulary. <img src="https://firebasestorage.googleapis.com/v0/b/cortex---2020.appspot.com/o/articles%2FNOeYFOIUIuSsHKoOczC8%2Fglphqu0QHA34?alt=media">Figure 1 - Example of a bag of words. This representation is simplistic and expresses certain limits. Word order
is not taken into account at all. Nor is it possible to tell whether a word
appears more than once in the same sentence. Furthermore, if a word present in
the sentence has not been included in the vocabulary, we will not be able to
represent it. Finally, each word in the sentence has the same importance in the
representation. This is certainly the greatest of the limitations cited: in a
sentence, it is obvious that some words, such as nouns and verbs, convey more
information than others, such as determiners.<o:p></o:p> To overcome this problem, more complex statistical representations have
been proposed. The most common is called TF-IDF (Term Frequency-Invert document
frequency, which can be translated by term frequency). This method is a
weighting method used to assess the importance of a term contained in a
document. This weighting is calculated using the frequency of the term in the
document compared to its importance in the entire corpus, that is to say the
set of documents that will be used as a representative sample of the language
to be transcribed. In order to fully understand what a document and a corpus
are, here are some examples: a document is La Fontaine's poem "Le Corbeau
et le Renard" (The Raven and the Fox), while the corpus is the whole of
his work., while the corpus is the whole of his work. Thus, a word that is very
frequent in a document, but also very frequent in the corpus, carries little
information: it is probably a determinant such as "the"
or "it". On the other hand, a word that is very frequent in
the document but less frequent in the corpus will certainly carry a lot of
information and will allow the subject of the document to be fully understood.
The TF-IDF method makes it possible to compare the frequency of a word in the
document with its frequency in the corpus, and to give it more or less weight
depending on this ratio.<o:p></o:p> This method is very efficient and is still used today in text mining.
However, this technique also suffers from limitations, notably the fact that no
importance is given to the meaning of the term. Its representation is purely
statistical, based solely on frequencies of occurrence of the term. No
information about the meaning of the term is therefore included in this
representation. Not yet creative
but constantly evolving<o:p></o:p>As in many areas of computer science, the representation of language is also being revolutionized by artificial intelligence. The first model that revolutionized this field is known as Word2vec, a vector word. Its principle is
 as follows: an artificial intelligence model, and more precisely a neural
 network will be trained to represent a word according to the use made of it in
 a training corpus.In practice, suppose that we want to train a Word2vec model in order to teach it to represent the words in the French language. The process will be as follows: we will provide it with a large number of texts in French, like all the articles in the French Wikipedia. The assumption that the model will follow is that two words used in a similar context have a similar meaning. The ultimate goal of the model is to produce a vector of word representation that translates the meaning of that word. Thus, two words with a close meaning will be associated with vectors whose values will be close.To illustrate this concept, the model must produce representations for the words
 in such a way that king - male + female = queen.The above operation is performed on the vectors representing each of these
 words. <img src="https://firebasestorage.googleapis.com/v0/b/cortex---2020.appspot.com/o/articles%2FNOeYFOIUIuSsHKoOczC8%2FVzfW1JXwmfuM?alt=media"> Figure 2: Illustration of the working principle of the
Word2vec template. (Source: kawine.github.io/)<o:p></o:p><o:p></o:p> 

Thus, the Word2vec model will allow us to produce for each word a
representation that has been learned by studying the context in which the word
was used. Each word is therefore associated with a single vector, this vector is supposed to translate the set of meanings that the word can take. In
practice, this is not exactly the case. Indeed, this static representation is
necessarily biased by the frequencies of occurrence of each meaning of the
word. For example, “wind” is more often used in the
sense of the physical phenomenon than the verb, “wind the clock”. Thus,
the model, which has been more often confronted with one meaning than another,
will tend to produce a vector reflecting the first meaning instead. This
limitation can be overcome by the dynamic representation of words or documents.<o:p></o:p> 

Dynamic word representations, which represent the current state of the art,
are revolutionizing many aspects of language processing, such as the
understanding of queries by search engines, translation, which is now carried
out on the Internet. sentence scaling or spelling correction. These advances
allow computers to grasp some of the richness that human language represents,
sometimes even exceeding their performance for certain tasks. While creativity
is not yet one of their capabilities, it is certain that the revolution is
underway in many aspects of communication.<o:p></o:p> Sources : 
1. JONES, Karen Sparck. A statistical interpretation of term specificity and its application in&nbsp;retrieval. Journal of documentation, 1972. 
2. DEVLIN, Jacob, CHANG, Ming-Wei, LEE, Kenton, et al. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. 
3. MIKOLOV, Tomas, SUTSKEVER, Ilya, CHEN, Kai, et al. Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems, 2013. 
4. https://kawine.github.io/blog/nlp/2019/06/21/word-analogies.html<o:p></o:p> 

 <o:p></o:p>

Category's article : "Computing science"