Recent Posts

Most Used Categories

AI, Journal

Containerize Everything

Reading Time: < 1 minute

In an effort to curb my video game playing hobby, and be close to my wife, I have been trying to stand up a bunch of machine learning services on my home server. I tried to do some standard things like WordPress and a regular website, but this proved to be inconsistent and tedious to configure. I decided to just pay for those services. But I’m designing more ML/AI oriented projects such as how to design a computer learning system similar to the human brain. And from previous experience, it seems that docker-compose is a good choice. Both for its ease to configure (someone did the job already in a docker container), and for modularity (I can start/stop a server without destroying my Linux server). So I decided to just use the latest docker images for things like MongoDB. The idea is to have each container do one type of service, and share the docker volume to a host/local folder for easier replication when I move to a different server. For this MongoDB container, I mostly followed the guide at this

dev.to tutorial

The only change I made was to change the volume to a local folder rather than docker volume so I can access it easier with greater modularization. Obviously this is not for production usage, but it’s sufficient enough for my own usages. 

version: '3.7'
services:
  mongodb_container:
    image: mongo:latest
    container_name: mongo_db_main
    environment:
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: rootpassword
    ports:
      - 27017:27017
    volumes:
      - ./mongodb_data_container:/data/db

Now you can use this to access the Mongodb.

mongo --username <username> --password <password> --host <ip address>

by yangliu3456
Misc

Covid-19

Reading Time: 5 minutes

There is no escape as this virus has dominated everybody’s life this year. I will write my personal journey here. And if in the future I get to look at it again, it may be a reminder for if I fall into similar situations again.

I first heard of the virus on Wechat, a Chinese social app, when a group of people at Wuhan thought SARS came back. There was some news about possible new cases of SARS, but it soon got quiet and nobody was mentioning it. I never gave much thought about it since similar rumors are quite often on social platforms. This was around mid-December of 2019. About two weeks before the Chinese New Year 2020, a bunch of people in China got tested positive for some new disease that’s similar to SARS. And there were some cases of death. It seems that things were starting to get serious. Then just before the Chinese New Year celebration, the government started to be open about the disease, and the New Year celebration TV show was filled with the topic. It was around how medical personnel was pulled from other areas of China to support Wuhan. My parents in law told me about how their activities were being limited. My brother-in-law and his wife found some surgical mask for my parents-in-law and send it to them back in China. It’s all-around Chinese social media now. This was just the end of January 2020.

At the same time, Americans were not very worried. My coworkers were not talking about it as much. I tried to warn them, but I don’t think anybody was actually concerned. I had to take a company-wide meeting in the last week of January. I took some surgical masks with me on the flight but was too “scared” to use it because nobody else was using it. In the U.S. this is a sign that someone is really sick. I didn’t want to raise that flag, especially since I’m Asian looking. On the flight back, my transport from the airport to near my house was a bus ride. On the bus ride, there were two groups of Chinese travelers wearing masks. They mostly have surgical masks, and one guy even has an N-95 mask. Even the two children had masks on. I remember I was thinking that my son will never put a mask on. At that time, most people didn’t know that children under 12 are not heavily affected by the virus. I had some debate about whether I should have pulled the masks from my luggage, but it was under the bus already, and I didn’t want to ask the driver to get it for me. I intentionally shipped two rows of seats as I passed them. They weren’t coughing, so I wasn’t too worried. That may have not been a good thing, because we now know it can spread without symptoms, even if they had masks on.

After coming back from the business trip, there was no change in terms of public sentiment. There were no known cases in America except the Princess Diamond Cruise travelers. President Trump did close the border to China pretty early. But there was no mentioning of any country in Europe, and no known cases in Italy yet. I pretty much carried out the same at work. No masks or gloves. But every morning, I would ask Google Home to tell me about cases near my home. And see if there are cases. I checked the CDC website pretty regularly. The Sina Chinese site was keep tracking of cases by day. And there are more cases every day. No media other than CDC is covering this in America.

Right around the time my son had a break in February, there started to have cases around us because they went to Italy for vacation. There are more confirmed cases due to travel around my hometown. I was getting increasingly worried. We discussed as a family and thought about whether we should take my son out of school, even though he will be marked absent. We did not end up doing it. I just watched super closely. However, when California, Washington, and Massachusetts starting to have double triple-digit cases, I worked up the courage to ask my boss to let me work from home. My boss was of course super understanding about it. The funny thing was our CEO decided my whole company is going to work from home the next day. So all that was just for one day of working from home.

Once I started working from home, we were more worried about getting food and try to limit our trips to crowds. We started getting deliveries more and stocking up on rice, flours, and other non-perishable foods. Trips to grocery stores are getting more intense. We used masks and gloves even though no more than 10% of the public is doing that. My coworkers didn’t seem to be too worried about it. We were, of course, trying to get masks. But they would either be extremely expensive, like $20 each. Or simply not in stock for physical drug stores. All the big vendors were out. Since my wife is close to the healthcare system, she was still working. They are starting to lock PPE supplies at work. She had two 3 more weeks than me because she simply cannot work from home. When she no longer required to work, she was put on furlough. Unlike my company, her employer didn’t ever plan a similar situation, or have the funds to support her. She was told to apply for unemployment benefits.

Life post the big event is settling in place. We don’t go out. Try to get food delivered to us when we can. And only go out for groceries when it’s necessary. I can still work from home, so it’s somewhat regular. I have done online meetings before so I don’t think work is hugely affected. My son has to study and do homework for about 6 hours a day. All that time is managed by my wife, so she is getting stressed more than normal. We go out to run early in the morning to get some exercise. We live in a somewhat rural town, so it’s no difficult to avoid neighbors. We just say hi from far away. I don’t know when I will be going back to work physically as my son’s school is “home only” for the rest of the year. I don’t think we can manage my son’s school work with both my wife and I working the whole day without teachers. There are some talks already for people going back to work because some states have protested about it. And I think people area unrest because not everybody can make ends meet. A lot of people need to work to survive. And some people are willing to take the chance to fight for what they built with their whole life. They don’t want to see that go away because of the virus. For myself though, I think I will stay put for a while until there is a definite plan to return.

by yangliu3456
AI, NLP, Reinforcement Learning

Using reinforcement learning for Conversation understanding

Reading Time: 3 minutes

I went to the NeurIPS 2019 conference in December and focused on NLP and reinforcement learning (RL) topics. The former is what I do for work, analyzing call center conversations, understanding what works for customer interactions, and make suggestions to clients based on their data. The latter is my personal interest, started all the way back when DeepMind beat the world’s best Go players. At the RL sessions, the tutorials mentioned using imitation learning to do natural language understanding tasks and generate responses to questions or chit-chat. People have some success, but it has some of the common pitfalls like repetitive usage of the most likely responses, and responses being too short. And sometimes the response is too simple and the bots fall into a cycle of “I don’t understand what you are saying”. So one night at a social gathering, I did get to meet Drs. David Silver and Richard Sutton. They briefly mentioned that I could try to set up the RL environment like a conversation and see if the agents can learn from the conversations. And in one of the workshops related to NLG conversations, people have talked about using various rewards to penalize for repetitiveness and encourage the generation of different texts. So that got me thinking. In addition to making a chatbot that’s similar to a call-center agent and customer interactions, I can design an environment that helps me discover the reason why people are calling. What I can do is set up categories of actions that are similar to “give refund”, “cancel service”, “keep the customer on the phone” etc, and use (regret = current action – best action), when the reward is either made the customer happy for 1 or made customer mad for 0. This may help me find the best action in a specific situation that results in the most wanted outcome given by the client. Granted, this might not give me the causal reasons why did the customer call directly, but it’s more similar to me designing a potential causal relationship graph before making the model and test if the causal relationship is correct. Even if it does not give me the exact cause, if it gives me the best action to take, then at least I have a product that does what the clients want. So the goal of the new year is to design an environment so I can test this idea and see if I can get the best type of action to take.

To get myself familiarize with the Gym framework from Open.Ai, I set up my own card game environment and made a double deep q-learning network. So here are some resources if you want to start your own:

Taking some design hints from https://github.com/zmcx16/OpenAI-Gym-Hearts
Some more help to design the environment: https://datascience.stackexchange.com/questions/28858/card-game-for-gym-reward-shaping
Other projects using gym: https://awesomeopensource.com/projects/openai-gym
Probably already programmed here: https://awesomeopensource.com/project/datamllab/rlcard
Create your own gym environments: https://github.com/openai/gym/blob/master/docs/environments.md https://stable-baselines.readthedocs.io/en/master/guide/custom_env.html https://github.com/openai/gym/blob/master/docs/creating-environments.md https://towardsdatascience.com/creating-a-custom-openai-gym-environment-for-stock-trading-be532be3910e https://medium.com/@apoddar573/making-your-own-custom-environment-in-gym-c3b65ff8cdaa

For the environment I look for in the conversation task, I will need to have an environment that mimics a conversation. A reset would result in starting a new conversation. Render is just how the conversation carried before a specific point. Each step can be a turn of the conversation, with texts randomly chosen from the same category pool. A downside I can see with this approach is that the training data might not generate the best solution to the customer service session. It may just be the bare minimum to get to the desired outcome. But I hope that the fact I can design my own regret in an RL framework, I can penalize for things like the length of the conversation or sentiment/emotional outcome, while I’m trying to achieve the outcome of retaining a customer.

I have encountered a similar situation before when I was making a chatbot using Rasa. They have a simpler RL environment, where a user can choose which route to take in a certain situation. But when I used it, the policy was too simple and does not achieve what I want, especially not give a causal relationship. I hope this could be integrated into this framework and be more useful.

by yangliu3456
AI, NLP

Combining multiple modalities of inputs in Neural Networks

Reading Time: < 1 minute

Recently, we have been trying to identify the emotion or sentiment in a phone conversation. The first step we have tried to use a while back was Universal Sentence Embedding. That algorithm transforms a sentence into a fix length sentence embedding or 512 numbers. But we are ignoring the acoustic features all together with this approach. We are getting 0.70 on test accuracy on about 2000 sentences/utterances, but hoping to do better. So we started trying to combine text embedding with acoustic features such as MFCC and may others offered by Librosa. I used an LSTM with features for each window as the time steps over however many windows I have. Of course, I had to pad to the maximum number of windows. I tried to even include all the features librosa offered but was not able to increase the accuracy of more than 0.73. After the concatenation layer of text and audio signals, I played around with both LSTM and Dense layers, with just dense layers being more accurate. I suspect there just aren’t enough samples, especially consider the LSTM for the acoustic signals. So will try to see if I can get more labeled data for the audio signal to improve accuracy.

by yangliu3456
AI, Reinforcement Learning

Introduction to Reinforcement Learning 2

Reading Time: < 1 minute

So continuing the journey on learning RL, I went through the deep Q blog.

https://www.freecodecamp.org/news/an-introduction-to-deep-q-learning-lets-play-doom-54d02d8017d8/

Now this time it required more setup on the colab environment. First, you got to make sure you can install the VizDoom package on colab. There are some dependencies needs to be satisfied. Also make sure you install scikit-image package. This takes about 10 minutes as the package need to be build using CMAKE.

Then I initially just pulled the stock “basic.cfg”, “basic.wad” file of the github site. But found out later that had problems. The big one being the frame was sampled in RGB and give you the wrong dimensions. So now I just download them from the github with the already setup config files. Nevertheless, it helped me to understand what was passed around.

A big thing to learn was, of course, the Deep Q network. Which is in the class “DQNetwork”. I’m more used to program in Keras, so it takes a little more time to understand. You can do this in Keras, although you need to update loss function, which is actually easier to understand to in TensorFlow.

The rest seems straightforward, just let it run and learn. Here is the full colab link.

https://colab.research.google.com/drive/1gZM4pAfH4kroa_44gNYZEE8RDVMiO9cP

by yangliu3456
AI, Reinforcement Learning

Introduction to Reinforcement Learning 1

Reading Time: < 1 minute

As I discussed before, there are not at many good reinforcement learning material out there. But I found this great set of tutorial and I will share my journey of learning it.

Start reading these two set of of blogs,

  1. Blog Number 1
  2. Blog Number 2

At the end of the second blog, you will find jupyter notebook for the Frozen Lake tutorial.

https://github.com/simoninithomas/Deep_reinforcement_learning_Course/blob/master/Q%20learning/FrozenLake/Q%20Learning%20with%20FrozenLake_unslippery%20(Deterministic%20version).ipynb

Before you start the tutorial, you will like need to learn how the Gym environment works. Go to this link and read the super basic tutorial they have there. Note especially what are the component of each episode. Actually figure out what are the possible actions, and what does each value of the state means.

http://gym.openai.com/docs/

Here is the wiki for the basic parameters. https://github.com/openai/gym/wiki/CartPole-v0

Refer to the source for what “Discrete” and “Box” are. https://github.com/openai/gym/tree/master/gym/spaces

Run the code on Google Colab and see how it runs. Print out the variable for each episode and step. I made an example in case you want to follow. https://colab.research.google.com/drive/1oqon14Iq8jzx6PhMJvja-mktFTru5GPl

Run the deterministic state first and then stochastic. Now you know how to create the super basic, Q table.

by yangliu3456
AI, Reinforcement Learning

Reinforcement learning

Reading Time: 2 minutes

Ever since the Deepmine Alpha Go paper came out, I have always been fascinated by what reinforcement learning could do. In my opinion, it’s the closest branch of AI to Artificial General Intelligence. Because no matter what your goal is, as long as you determined the reward correctly, it will eventually help you to find the “optimal” solution. What it lacks, perhaps, is that it takes a long time to learn. For example, human babies don’t have to stumble millions of times to learn how to walk. And it’s even more evident for a deer, where they pretty much know how to walk after they were born. Now, that is not too say evolution didn’t give some prebuilt model for the deer to use, but maybe we can make some pretrained model for AI to know how to walk, like the modules they insert into Neo to give him fighting abilities.

When I set out to learn reinforcement learning, there wasn’t much formalized material. There is Sutton and Barto’s book, but I found it was and still written for experts. The exercises they give are necessary for me to understand what’s going on, but there is no explanation given. There is too much “for obvious reasons” for me to understand some basic concepts, like how do I calculate state value for each grid. There is a formula for it, but no detailed walkthrough to how to assign those values. Then there are David Silver’s lectures, which shined more light for me than the book, but there is also too much detail left unexplained. So, I have searched the internet for some tutorials and finally landed on this medium article. https://medium.com/free-code-camp/an-introduction-to-reinforcement-learning-4339519de419. It has enough little details for me to work out how to get to individual values, starting from what’s a Q table and how implement it in python. I highly recommend going through the tutorial and exercises on Google Colab and make sure you understand how the environment is used in the Gym package.

Anyway, I’m still working through all of the code and understanding it. I also recommend a Lego Mindstorm set to give you a physical manifestation to work with. It can help you familiarize with how to apply it in the physical world. The basic robotics with a stable, not bipedal, robot, also makes learning easier. Make sure you use a customize python package like ev3dev so you can control the robot with python scripts. https://www.ev3dev.org/ Now that’s how you have endless, time sucking fun!

by yangliu3456
AI

AI learning material

Reading Time: 2 minutes

Recently, I have received many requests about how I learned Neural Networks. Since this is so new that not that many Universities offer a class in it, I thought I should put together a list of learning material I found useful. Hopefully other people will find it useful too.

  1. I’m an audio learner and love to listen to people. So Andrew Ng’s Coursera course was good for me. https://www.coursera.org/specializations/deep-learning. Take all four if you have time and try to understand the exercises and not just manipulate the program to get the correct answer. Actually understand the Math behind it.
  2. Ian Goodfellow’s books cover the math background you need and he shows you how to apply it. https://www.deeplearningbook.org/. This is a good reference book if you find yourself lost in some concept a paper mentioned. The dreaded, “for obviously reasons, …”
  3. If you find you want some statistics background, I read this book, http://www-bcf.usc.edu/~gareth/ISL/. There is another more bible like statistics book, but if I cannot understand it, I’m not going to suggest it to anybody.
  4. Cornell University have an open archive for all sciences. And you can find the specific journal articles related to AI, Computer Vision, Speech, and NLP. https://arxiv.org/. It is intimidating in the beginning to read papers, just from the amount of jargons people use. But after you read several it’s not that bad. Otherwise, refer to Ian’s book or just google it.
  5. MIT online course videos are good for general background in Math and CS. Again, I prefer listening to lectures much more than reading books. I just fall asleep too much.
  6. Stanford class notes is where I started: http://cs231n.stanford.edu/. Image recognition is easier to understand but hard to master.
  7. Grant’s video on youtube has the best visualizations. And they are the best linear algebra video I have ever seen on transformation. You are going to need linear algebra. It speed up your computation by a ton, if nothing else. https://www.youtube.com/watch?v=aircAruvnKk&list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi
  8. I would say most import of all, find a paper with code and just play with it. You can put code on Google Colab and get free GPU. https://paperswithcode.com/

Secret: Use Twitter to follow the experts in AI. Build it up to specialize in the field you like. e.g. reinforcement learning. I will get you started with some famous names: @geoffreyhinton, @ylecun, @AndrewYNg, and @goodfellow_ian.

by yangliu3456
Misc

Incorrect password for Ubuntu login screen

Reading Time: < 1 minuteI have been having this problem multiple times, and there are so many non-working solutions online. So I thought I should recorded for future usage.

Edit the following file with this content:

sudo vim /etc/lightdm/lightdm.conf

vim can be replace with whatever command-line editor you have available, nano, emac, etc…

[SeatDefaults]

greeter-session=unity-greeter

greeter-show-manual-login=true

by yangliu3456
AI, Misc

How I went from a Neuroscientist to an AI practitioner – Part II

Reading Time: 4 minutesAfter studying further about what neural networks can do, I realized this is what people are going to do in the future. For everything. And I know I got to get a piece of it. So I started to learn the basics. At the time there was not much training or tutorials. AlexNet was widely used and I ran across a tutorial on using Matlab to make VGG16 convolutional NN. I had pretty good video card, but it wasn’t able to run the Matlab package. I had my eyes on the upcoming Nvidia 1070 cards. I had to wait for it to release and I new it was going to be hard to get.

In the mean time, I figured out it doesn’t have to run GPU. And I can start learning the basics before I can get my hands on the new cards. I searched around. Most of the tutorial have fragmented information, and I feel I have no grasp on the material and did not really understand anything. Then I found Stanford’s CS213n class notes, and felt I hit the gold mine. It was technical and theoretically involved, but it explained how CNN model pixels and many of the CNN tools like weight initialization and dropout.

Once I got a sense of how CNN works, I joined Kaggle’s competitions to actually ran my own models. I tried the simple ones like Titanic morality prediction that require heavy feature engineering, but quickly realized that’s not the way to do things, because it required too much manual work and introduced bias. Then I saw the competition where State Farm want to categorize pictures taken inside of cards for what drivers were doing. That required me to use opencv for image pre-processing and categorized pictures using training and test sets. I didn’t end up having a super high ranking, but I learned various ways of doing it from people’s published kernels.

Fast forward several month, and I got my hand on a Nvidia 1070. Popped that puppy in the tower and sped up the models by a huge chunk. Of course, by this point, I knew Python is the way to go and used Theano and Tensorflow for making the models. I played with some public data and started a project to estimate house price in NYC, because everybody was doing it. I tried out for an interview with Insight Data Science. I prepared my projects and seemed to know what I talked about. I was told that I was on the board line and barely missed the cut, but maybe she was just being nice.

No problem, I will try again. On the third real try for Data Incubator, I finally answered enough questions in Python and got an interview. After a group interview with 6 other applicant, I got an offer for the paid, second tier fellowship program. Supposedly, I wasn’t as good as what they are looking, but if I pay for the fellowship, I can tag along. It was a large amount of money, even for NYC standards, but I had other considerations.

My Professor was about ran out of funding, at least for me. So I was about to be out of a job, unless I want to find another postdoc position. I started looking for industry jobs, but wasn’t successful. Just one week earlier, I got an offer to join a doctor’s office to do In Vitro Fertilization stuff. I applied because I have many family connections in that field, and I actually know how to do the stuff. It was a safe job with limited upside. My wife encouraged me to turn it down and pursuit my dream in AI. And I thank her so much now. I would be doing the same thing day after day to earn a living if I took that job. But instead, I’m learning new things everyday and loving it.

Coming back to my decisions, it was tough to turn down offers when I about to be unemployed. Again, my wife agreed to “take care of me” for 6 month, since my research said that’s the average time of unemployment for changing careers. It gave me a piece of mind, and let me focus on what to learn. With all my personal affaires lined up, I accepted the layoff and prepared to be workless, payed a hefty amount for the fellowship, and doubled downed on my future.

The fellowship was enjoyable. Hard, but fun. We were squeezed in a box for at least 8 hours a day. Studied as much as we could, did a lot of programming, prepared for interviews, and met some potential employers. It was a good variety of topics, and I was still talking about it 6 months later. As fun as it was, time flies. But the end of the session, 2 people got offers. Some are hopeful, I wasn’t one of them.

When the time of workless came, strangely, I wasn’t afraid. Moved out of NYC, we ranted an apartment in a suburban area. I kept the discipline of applying jobs everyday and interviews kept happening. A few showed interest, but ultimately, none of them went through.

Finally, a family member helped me to find a job, but it a rollercoaster. I got a phone interview a year earlier. A statistician asked about what models I made before and some other technical questions for half an hour. No feedback or anything after that. Seven months later, I got an email about scheduling onsite interview out of the blue. Of course I went. It was a grueling 8 hour day interview. Mostly behavior questions, with a few stat questions throwing in between. Then nothing. Mostly mentioning there is no position opening right now.

A month later, I got an email from a different department asking to schedule a phone interview. This time more questions on neural networks and programming. I knew I wanted whatever job they have. But again, no feedback on job or no job. A month later, same group of people called again and simply asked do I want the job. They knew I wasn’t sold on programming, but they are willing to teach. I had to go for an onsite, but from what I felt, it was just a routine I had to go through. The decision was already made.

Everything was what I though it would be and better. The rest is history. Now I’m working on applying the latest AI algorithms to industry related tasks. Some people asked, do you feel overwhelmed by doing more than 20 projects in one year. I responded to that: I was never too busy to do the things I love.

So, for people who are looking to change their career to AI, or any career you love, don’t give up. The road may be long and scary, but we only live once, and you should go for broke.

by yangliu3456
View More