Automatically Better

Last year, when news emerged that social media giant TikTok might go up for sale, some estimates put the company’s valuation as high as $60 billion, or almost twice the net worth of Twitter. One asset was largely responsible for turning it into such a hot commodity: the closely guarded algorithm generating TikTok’s recommendations, the secret sauce its parent company, ByteDance, decided was too valuable to sell.

Artificial intelligence has transformed business analytics. It informs how companies run the electrical grid, handle shipping logistics, and manage investments in other companies. New algorithms can determine how social benefit gets factored into the equation. They also can make or save billions of dollars.

Say a company wants to route trucks from a warehouse to a store or give app users directions home. These are examples of a “shortest path” problem. Finding the fastest journey through a sprawling web of streets under changing conditions is daunting. Adam N. Elmachtoub, associate professor of industrial engineering and operations research (IEOR), tackles such problems by combining machine learning with operations research. The former typically uses math to make predictions, while the latter uses math to make decisions.

In a standard approach, machine learning would predict the travel time for each link, then operations research would use the predictions to find the shortest chain. But some prediction inaccuracies can have disproportionate effects on decisions. So Elmachtoub trains the machine learning model, not to make accurate predictions per se, but to make predictions that lead to good decisions. His technique can also improve financial portfolio management, predicting companies’ performance in a way that maximizes portfolio profit or minimizes risk.

Ideas from portfolio management, in turn, can benefit other areas of business analytics. One area is clean energy. New York State wants net-zero carbon emissions by 2050. “So how do we get to that very ambitious goal?” asks IEOR Professor David Da-Wei Yao. Yao is developing strategies for clean energy deployment in the way one might assemble a collection of assets. Solar, wind, and nuclear each have their own risks and rewards, just like investments.

Yao also hopes to plan for a cleaner future by combining machine learning with economic simulations. One approach is to look at historical data to learn the effects of incentivizing solar power or taxing dirty energy on the market, then plug these associations into a simulation and turn the crank to forecast the future. “It’s almost like doing an experiment to predict the outcome,” he says.

Yao will pursue both projects as co-director of Columbia’s new Fintech, AI, and Business Analytics (FABULYS) Initiative, an effort designed to work closely with industry.

Some types of machine learning do more than make predictions; they can also make decisions. Shipra Agrawal, assistant professor of IEOR, uses reinforcement learning, in which a program makes sequential decisions (such as chess moves) and improves based on eventual costs or rewards (winning or losing a game). While learning, the program balances exploitation (doing what it thinks is most advantageous) with exploration (trying something else in case it’s even better). One application is inventory management. General reinforcement-learning algorithms struggle to explore the large space of possible actions, but Agrawal has found that by building in certain assumptions about inventory managment, learning happens much faster.

Algorithms can even learn to create other algorithms. Just a few years ago, Agrawal wondered, Is it even possible for a program to design programs to solve problems? It is. Many programs that manage production schedules, shipping routes, or inventory management require experts to program rules of thumb and to hand tune hundreds of knobs. Agrawal uses reinforcement learning to do that automatically. It’s still a proof of concept, she says, but it could be combined with the heuristics-based approaches.

As AI increasingly makes decisions for us, we need to think about how to apply it ethically. Operations research can help. Elmachtoub has looked at situations where different customers pay different prices— for instance, for college tuition. He’s found that imposing fairness may or may not increase social welfare, in terms of total benefit to both producers and consumers, depending on whether fairness is defined relative to price equality or access equality.

Similarly, Agrawal has formulated fairness trade-offs, in which fair decisions that cost a company in the short term might draw customers in the long term. Fairness and profit can co-exist. The trick, she says, is to “look at fairness in a more holistic way. Maybe the way we discuss trade-offs will change.”