Blog

IIITH word embedding workshop

Jun 13, 2019 | 3 minutes read

“The syntactical nature of reality, the real secret of magic, is that the world is made of words. And if you know the words that the world is made of, you can make of it whatever you wish.” - TM

IIIT Hyderabad

Invitation.

From May 13 to June 4, I was heavily invested into figuring out what the current SOTA language models were in the NLP domain in order to implement a paper on Poetry to Prose conversion, a task given to me by Amba Kulkarni Ma’am, in HCU. My mind was intensely filled with all concepts clustered around the topics of word embedding like word2vec, gloVe then leading on to language models like ELMO, GPT2, BERT and so on.

The uncanny coincidence was, just then I got a mail from Amba Ma’am on June 4 stating that, there is a workshop being conducted by Prof. Vineet Chaitanya in IIITH from June 6 to June 12th.

How could one say no to such an opportunity. We found ourselves to be in IIITH on June 6th.


Workshop. Enter.

We went to Vindya A3 building to Prof Vineet Chaitanya’ s cabin. There we were introduced to many Computational linguistic and Sanskrit Scholars.

Some were professors, in other universities (Chinmaya Vishwavidyapeeth University) , some PhD scholars from IIITH itself, some visiting PhD students from Gujarat and UP.

The task proposed to us by Vineet Sir, was to take leverage of the interdisciplinary mix of people we have in this workshop setting and work our way out to get a intuitive understanding of Word embedding as a whole, by discussions, presentations and note taking.


Workshop. Start.

The below diagram depicts a day’s routine.

Day 1 :

Our workshop started with a discussion on linguistics aspects of a language, specifically on Sanskrit. Defining key concepts such as Karma, Karta, Karaka, Fal and many more from which we could segue to the topics of word embedding.

Day 2 - 6 :

From the day 2 to day 6, we tackled, dissected a host of topics from scratch.

The topics were:

  • Linear Algebra
  • Vectors
  • Matrices (their role in transformation)
  • Eigen values and Eigen Vectors
  • Dependent and Independent Vectors
  • Diagonalization of Matrices
  • Linking above topics with Word2Vec
  • Word2Vec model
  • Limitation of Word2Vec
  • Current SOTA models Like GPT-2, BERT

Workshop. Takeaway.

Why the workshop was conducted in the first place one might question ?

It solves various concerns at once.

  • Multi-disciplinarian approach to a topic.
  • Ignite different way of thinking. (paradigm change)
  • Learn new insights. Novelty.

However, this was the first exclusive workshop that I attended and the major takeaway has been to observe and imbibe as much as possible the mental models of Prof Vineet Chaitanya.

He is such an individual, who through his vast expanse of experience on a panoply of topics, just sees through what approach of thinking is being employed and where it might lead an individual to. His primary emphasis is on thinking in first principles, hardly any one these days tries to instil in oneself and others a lens view to approach things in that manner. In this workshop, although it was a bit difficult in the beginning to operate under the first principles approach, while presenting ideas however the satisfaction one gets after, going through the ‘aha moment’ as sir says, and one feels is ineffable.

Glad to have received the opportunity from Amba Kulkarni Ma’am, Prof Vineet Chaitanya, to explore and tweak thine mental thinking and meet the other curious, diverse attendees.

Workshop’s last day pic with Guruji and other attendes. Arun Ji and Soumya Ji missing in this photo.

comments powered by Disqus