What is so hard about natural language processing? Hear Expect Labs' NLP expert, Marsal Gavaldà, explain.
What would be the hardest aspect of natural language processing? Well, precisely because computers don't have a body; they don't physically interact with things, it's a bit more difficult to teach a computer to fully understand the meaning. However, this is being solved with the existence of a knowledge graph, which is a giant network of concepts and relations. That helps anchor the results, and allows for things like disambiguating, so i I'm using "play" as a verb or as a noun. Also, for the so-called Anaphora Resolution, which is when I hear a pronoun, so: "Can you play a song by Bruce Springsteen? Do you have anything else by him?" This "him" now refers to Bruce Springsteen in this context, but in the general case it sometimes could be quite difficult to establish the proper reference to a pronoun.
Then of course there's the usage of the language, because sometimes we can be quite ironic. If I say, "yes, that's correct," that's very positive. "Yes, that's right," also positive. If I slightly change it and say "yeah, right," now it's no longer a positive, in fact, it's a negative. You have to be attuned to these cues, which are sometimes prosodical, just based on the intonation, and most times based on the full understanding of the context.
These are some of the aspects that make it hard, but as we progress and we have more data, we're also getting better at constructing complex algorithms, for example using deep learning. We are able to build computer systems that get smarter and smarter, almost to the point of having humanlike abilities to process natural language, to understand natural language