The seminar given by Prof Wray shared some non-parametric methods for unsupervised modelling based on an example in NLP (natural language processing). With current growth of information systems, we need some better tools to help us to deal with the information overload. Such tools should be capable of organizing, searching, summarizng and even understanding the information. The 'understanding of the meaning of natural language' or in other word 'semantic' is the main focus of Prof Wray's methods.
The unsupervised learning in this model is based on Dirichlet distribution from probability and statistic. Throughout the presentation, Prof Wray tried to avoid the complex maths functions he has used. Basically, the concept involves probability vectors for the following elements quoted from Prof Wray's note:
- the next word given (n − 1) previous,
- an author/conference/corporation to be linked to/from a webpage/patent/citation,
- part-of-speech of a word in context,
- hashtag in a tweet given the author.
A Dirirhlet distribution is then used to develop Dirichlet processes for the semantic model. It enable the approximation of vocabularies or documents hierarchically. The benefit of this model as per said by Prof Wray is that it reduces parameters optimisation problem faced by most distribution functions. Also, the nested (or hierarchical) Dirichlet processes have fast samplers as compared to others.
This is indeed a very high level of learning for me as the concepts and functions involved are really something new to me. Anyway, thanks to the seminar, I am more open to some new and advanced algorithms for my research project.
No comments:
Post a Comment