“How many piano tuners are there in New York City?”
When I read with shock how ChatGPT was able to pass the U.S. Medical Licensing exam, a Wharton MBA course exam and a University of Minnesota law school graduate exam, I decided to pose it this question I was asked earlier in my career when I was interviewing with Deutsche Bank in NYC for an Associate Analyst position. This problem is designed to test someone’s deductive reasoning skills as what matters is the logic used to reach a reasonable estimate, not the answer itself. ChatGPT failed this test as it responded: “it’s estimated to be around several hundred” and when I dug further and asked it “how would you figure out how many there are?”, it suggested: “one could conduct a survey or gather data from industry associations, yellow pages, music schools and local tuner directories.”
UPDATE: I posed this question to ChatGPT when I was proofreading this book in early July 2023, and as it was now powered by GPT-4, it came up with the correct response, starting with: “This question is a classic example of a Fermi problem, named after physicist Enrico Fermi. The idea is not to know the exact answer, but to use logical reasoning and estimation to arrive at a plausible approximation.”
Microsoft’s deployment of OpenAI’s tools like ChatGPT across its suite of enterprise and consumer products poses not only the threat of structural disruption to Google’s advertising-driven search business model but also of increased competition to Amazon’s AWS. In a broader scope, the real disruptive power of AI tools like ChatGPT lies in its economics. Whereas it is highly capital intensive and takes time for companies to deploy AI robots across their factories, warehouses and stores, AI tools like ChatGPT are in the cloud so they have zero cost of installation, their marginal cost is very minimal and they can be deployed instantly. So whereas AI on the physical level faces the economics of scarcity, AI on the virtual level benefits from the economics of abundance.