Panzoto

Recently, I’m starting to send my son to day care. Inevitably, it takes a while for him to adjust and he cries when leaving the house. Just like my older son, he is smart enough to know that if he leave home at that time and with me, he is going to daycare.

And then I started to ask a question. Would he cry as much if he was sleep trained? In another words, is the total amount of crying the same for a child? Would being sleep trained harden him, and make this transition easier? I don’t know the answer to this question. And I cannot test it either since I can’t go back in time. Even if I can go back in time, my wife would never let me know sleep train because she needs to hear the crying. We are only okay with this because he will be in the car with me during drop off, and I’m the only one that will here him cry.

On a happier thought, I think it give a bonding moment because I have to go through this life changing event with him. And from what I heard, people bond during tough experiences, e.g. school, military, etc. I went through this with my older son. He wasn’t attached to me much when he was little, but as an older child, he is more attached to me than his mother. If this continue to hold, there is at least a purpose for me to go through this event with my younger son. Because I know we will bond over this. My purpose as a parent in life, is to make sure he can be on his own when he grows up. And I believe I’m taking the first step towards that.

The Biden administration just released an executive order on October 30th, 2023 regarding the governance of artificial intelligence. I applaud the effort the white house is making towards regulating the AI industry and protecting the general public regarding privacy and and safety concerns. These efforts focuses on national security issues regarding biological, chemical, and nuclear weapons, specifically associated with biotechnology and cybersecurity. It calls on the respective federal agencies to draft plans in the next 90-270 days to regulate companies who are involved in the AI industry.

There will be a lot of good things coming out of this executive order. But as human nature dictates, it’s just easier to criticize than to praise the positives. So here is what I think that needs improvement:

1. My major concern is that this document will prevent the entry of small companies to the large language model (LLM) market. Early in the document, one of the principle for the guideline was to promote competition, especially from the smaller companies. But later in the document, it calls on the agencies to draft guidelines that require companies to report the development LLMs over 10 billion parameters. The intention is to keep track of the models that can be used for many purposes, including those that pose national security risks. However this requirement inadvertently impede the development of models by small companies because they wouldn’t have the ability to generate the documentation required by these agencies. This 10 billion parameter limit means the latest llama open source models is included in the reporting requirements, including companies that simply possessing these models. This mean the making of large models are left in the hands of the tech companies like Google, Facebook, Microsoft, etc. This is not what the general public like to see.
2. Another thing I see is that some of the guidelines and policies are difficult to enforce. For examples, one of the principles is reducing the effect of AI on general workforce. It is a well intended principle. But in the practice, profit seeking nature of companies means that they will use the power of AI to reduce workforce spending. Even though many companies talked about “augmentation” of workers, it’s a fact that the worker count have been reduced because of AI implementations. The reality is that for people who cannot or unwilling to learn about AI technologies, they will be impacted by workforce reduction in the future. And this future could be sooner than people think.
3. It’s late. Regarding privacy laws, we are behind Europe on GDPR requirements. Some states have their privacy laws, such as California, but we still don’t have unified national requirements. For comparison, China started to mandate online commerce companies to stop using consumer data to track and target consumers with the sole purpose of maximizing profit. One example it to prevent the practice of giving new customers better deals while “ignoring” old customers. Or selectively give people worse deals because the companies knows that they will purchase such items regardless.

Over a month ago, I was back on the job market. I connected with my friends and here are the opportunities I have found and a few tips on what the market is looking for.

1. Products that use search. I met a lot of knowledgable individual in this area and learned a great deal along the way. Needs in this area revolves around using traditional search algorithms and mix in neural network approaches as well. I was surprised to find that keyword-based algorithms like BM25 is still one of the widely used methods. BM25 is similar to TFIDF, but rather than just count the rare words, it is also divide the frequency by the length of the text. So the longer text were not given favor just because there are more words in it. Even though they don’t have semantic meaning, this algorithm is still among the top performers of BEIR scores. I was surprise to find this since I have used similar algorithms in topic modeling, but found the results to be much inferior to using sentence embedding. The group I talked to in this area is more concerned about implementation of personalization of the search result. Many of techniques require “augmentation” of the search query. For examples, using intent classification on the search query to generalize it’s purpose. Since many search query are short phrases that barely have context, the algorithm essentially need to guess what people are asking for based on the personal history, location, and what other users found to be useful. Other preprocessing including segmentation of the search context, so larger groups are identified. The groups are basic units to be search rather than individual documents.
2. Fine-tuning of large language models. Since GPT4 and large language models is the current and hot topic, many companies are looking to take advantage and ride the wave. It’s not always the best solution for their business case, but companies do occasionally find it useful. More often people use out-of-the-box LLMs to gain knowledge to their documentation. For example, if we want to able to search through written documents and find relevant answers, we can simply put that large document in chunks through the GPT4 model and ask for an answer. Or we can use LLM to build embeddings and search query against the embedding using cosine similarity and rank the results. Up until recently, GPT4 models have no access to internet data to fact check. Even if they do, they might not know business specific knowledge. So there are packages help user search through those information as well. Algorithms like Retrieval-Augmented Generation and Low Rank Adaptation are particularly useful. A major advantage of this approach is that they don’t require a lot of data to fine tune the large language model.
3. Causal inference modeling. This type of analysis is often for industries that often need to figure out the root cause. One example could be the insurance industry. For example, if I want to figure out the important factors that influence the price of auto insurance, I can simply run a general linear model with feature like age, marital status, etc. to predict the insurance premium. And use models with feature importance to figure out which feature influence the premium the most. But this often does not solve the business problem since the executives will question why they think a particular feature is important. So the data scientist not only need to figure out the what the factors are, they also need to find out how. One way people do this is by constructing equations they think how these features influence the premium. This is often based on their industry experience and intuitive sense. I cannot say I understand this part, so it seems that many years of experience is need.
4. Company-wise tool development. In larger corporations, there are research teams that build machine learning tools aimed to help the whole company. Since ML talents are rare for any company, they are looking to maximize the value. Instead of targeting specific use cases, they are asked to develop generic tools. This is actually the most common theme I have seen for the jobs I applied to. The needs range from fresh new departments to seasoned corporations with years of ML research experience. The only difference is that newer teams focus more on tool building, whereas mature teams thing more about large scale production and data normalization.
5. Specific product development. As apposed to company-wise tool development, many company have specific productization goals. They may or may not include ML specific component, but I tried for these positions anyways. Most of them are not the best fit for me. Some are because of ethical reasons, and some simply because I’m not that good of a programmer and they don’t need ML knowledge at all. All and all, I’m glad I tried just to see where I’m and what I don’t want to do.

At the end, my decisions is to go with a position that’s most comfortable for family reasons. That same role also have enough ML needs and I can use my past experience and explore new things at the same time. To sum up the learnings for this experience, is to get as many options as possible, and really figure out what I wanted to do. Don’t shy away from tough interview questions because sometimes they help you figure out life purpose as well. And for offer negotiations, having multiple offers make it a lot easier to play the “don’t tell your salary expectation” game. It helped me immensely since I don’t have natural game to get the max offer I can get.

Expanding on using the Decision Story tool to help people make decisions. I want to explore the decision making process of business executives, and how this tool could help that target audience. By talking to people who are in the executive role and interacted with CEOs, I found one popular way for which CEO uses to make decisions. For a decision that may change the direction of the company, CEOs would like to ask the alternatives, and how these alternative changes the financial bottom line, as well as departments of the company. They want to know what would change if they make the decisions in as many aspect of the company as possible. So they could evaluate if those changes is good for the company overall.

So I though, what if the decision making tool could incorporate those aspect and ask the large language models (LLM) to evaluate it through the lens of an Econ. An Econ is often referred to in the economy academics as someone who always make the most logical decisions with the lack of emotion and bias. I have tried to use this approach and got some reasonable answers from the LLM. And I have found the answer covered some aspect of the decision that I haven’t realized. There are definitely CEOs who are smarter than me, but it probably doesn’t hurt for people to know that they already covered the basics. Here are some of aspected worth mentioning when asking the LLM for alternatives.

Here is an example in which I asked the tool to help me make a decision to whether to use a in-house model, 3rd Party model, or a combination approach to improve an existing product.
Here is what I put in:

“I’m an CEO of a mid size company with 300+ employees. We are working on speech analytics and helping customer to understand call center conversations. Large language models are getting big. I have some decisions about which model to use. We developed some simple binary classification model in house. We can use Openai 3rd party models. Or we can use our existing rule based solution. Or we can use a combination of some of the above. Please provide the pros and cons about each and provide other alternatives for long term growth. Specifically aim at IPO in next few years.”

And here is things I’m putting into the models.

1. Objective Definition:

Define the goal you’re trying to achieve. In your case, it might be “select the best AI approach to enhance our speech analytics service.”

2. Information Gathering:

Gather as much information as possible relevant to your decision. This includes:

• Business Requirements: Specific needs of your business and customers.
• Technical Requirements: Scalability, data volumes, security, etc.
• Market Trends: What your competitors are doing, trends in AI technology.
• Legal and Regulatory Requirements: Compliance to data privacy laws, etc.
• Financials: Budgets, potential ROI, etc.

3. Decision Criteria:

Establish the criteria you will use to evaluate your options. This could include factors like cost, time to implement, ease of use, scalability, potential return on investment, etc.

4. Generate Alternatives:

Generate a list of potential options. In your case, these are the AI technologies you’re considering: In-house model, third-party model, rule-based solution, or a hybrid.

5. Evaluate Alternatives:

Use your decision criteria to evaluate each alternative. This may involve financial modeling, technical assessments, consulting with experts, and other forms of analysis.

6. Select the Best Alternative:

Based on your evaluations, select the option that best meets your criteria and aligns with your business objectives.

7. Action Plan:

Develop an action plan for implementing your decision. This should include key milestones, resources required, risks and mitigation plans, and a timeline.

8. Review Decision:

After a certain period of implementation, review the decision to see if it’s delivering the expected results. If not, understand why and adjust as needed.