Take a deep breath.

Now that I’ve made that everyday statement to you (or perhaps it is a commanding directive aimed at you), suggesting that you are to take a deep breath, what would you do next?

I suppose you could completely ignore the remark. You might brush it off. Perhaps it was just a figure of speech and not intended for your attention per se. On the other hand, maybe you interpreted the remark as quite helpful. Ergo, you have indeed stilled yourself and taken a deep breath. Good for you. We all seem to know or be told that taking a deep breath can be good for the soul and get your mind into a calm contemplative state.

Consider the catchphrase in a different light. If you were to be told that same remark in a medical context, such as a physician saying the same to you, I am assuming that you would be interpreting the words to mean that you are supposed to purely physically take a deep breath. This might have nothing to do with your mental status and instead, be part of a physical exam. The doctor might be listening to your chest and wanting to detect internal cavity sounds as prodded via a heavy inhalation.

The gist is that the famous line of taking a deep breath can be contextually sensitive.

You have to consider the circumstances under which the line is being used. It could be a command. It could be intended to get you to stop and smell the roses. It could be a physical thing. It could be a metaphorical indication that has some other significance. A wide array of possible interpretations exist.

The same can be said for using the take-a-deep-breath catchphrase in generative AI.

In today’s column, I am continuing my ongoing and popular series on the latest advances in prompt engineering, including that I am going to tackle in this discussion the recent trend or fad of wanting to make use of the line “take a deep breath” as a prompting technique or strategy for generative AI. You might not have heard about this as yet. The “take a deep breath” mantra is slowly gaining momentum. I thought it might be handy to catch this roller coaster on the way up.

My goal here is to share with you why this is not some magical incantation that will solve all kinds of prompting issues and get generative AI to work miracles. Sorry to break that harsh or gloomy news. But, hold on, there is a smiley face still to be had. The use of the deep breathing catchphrase can have some limited advantages in generative AI. I am just forewarning you that it is not a catchall and you must use it in only the most astute of ways.

I will proceed in my discussion as follows.

First, I will introduce you to the said origins of the take-a-deep-breath prompting approach. Doing so involves doing a bit of a deep dive into some recent research on the optimization of prompts for the purpose of boosting prompt engineering results. You’ll enjoy those aspects, I’m sure of that. The research is quite interesting, and I applaud the researchers for their insightful and innovative work.

Second, I will discuss why you need to be cautious in overinterpreting the properties of the legendary saying. It has been somewhat unfairly plucked out of the midst of a fuller research study and given a shiny light that seems to overstate its importance. Also, turns out that some in the media have nearly in their glee run somewhat amuck with the catchphrase. They appear to be touting it as a prompting savoir in ways that are misleading, misguided, or maybe naïve and innocently stirred without having intimately studied the matter.

Third, I decided to do some ad hoc experimentation to try and look more closely at the deep-breath conception as a prompting tool, doing so by using ChatGPT. Admittedly, the look-see is short and sweet. The quick-and-dirty effort was not a full-blown experimental setup. My guess would generally be that the tiny indication here might spur some erstwhile added AI research of a deeper analysis of the deep-breath considerations. Maybe, maybe not. I’ll let you know if something else pops up.

Before I dive into my in-depth exploration of this vital topic, let’s make sure we are all on the same page when it comes to the foundations of prompt engineering and generative AI. Doing so will put us all on an even keel.

Prompt Engineering Is A Cornerstone For Generative AI

As a quick backgrounder, prompt engineering or also referred to as prompt design is a rapidly evolving realm and is vital to effectively and efficiently using generative AI or the use of large language models (LLMs). Anyone using generative AI such as the widely and wildly popular ChatGPT by AI maker OpenAI, or akin AI such as GPT-4 (OpenAI), Bard (Google), Claude 2 (Anthropic), etc. ought to be paying close attention to the latest innovations for crafting viable and pragmatic prompts.

For those of you interested in prompt engineering or prompt design, I’ve been doing an ongoing series of insightful explorations on the latest in this expanding and evolving realm, including this coverage:

Anyone stridently interested in prompt engineering and improving their results when using generative AI ought to be familiar with those notable techniques.

Moving on, here’s a bold statement that pretty much has become a veritable golden rule these days:

If you provide a prompt that is poorly composed, the odds are that the generative AI will wander all over the map and you won’t get anything demonstrative related to your inquiry. Being demonstrably specific can be advantageous, but even that can confound or otherwise fail to get you the results you are seeking. A wide variety of cheat sheets and training courses for suitable ways to compose and utilize prompts has been rapidly entering the marketplace to try and help people leverage generative AI soundly. In addition, add-ons to generative AI have been devised to aid you when trying to come up with prudent prompts, see my coverage at the link here.

AI Ethics and AI Law also stridently enter into the prompt engineering domain. For example, whatever prompt you opt to compose can directly or inadvertently elicit or foster the potential of generative AI to produce essays and interactions that imbue untoward biases, errors, falsehoods, glitches, and even so-called AI hallucinations (I do not favor the catchphrase of AI hallucinations, though it has admittedly tremendous stickiness in the media; here’s my take on AI hallucinations at the link here).

There is also a marked chance that we will ultimately see lawmakers come to the fore on these matters, possibly devising and putting in place new laws or regulations to try and scope and curtail misuses of generative AI. Regarding prompt engineering, there are likely going to be heated debates over putting boundaries around the kinds of prompts you can use. This might include requiring AI makers to filter and prevent certain presumed inappropriate or unsuitable prompts, a cringe-worthy issue for some that borders on free speech considerations. For my ongoing coverage of these types of AI Ethics and AI Law issues, see the link here and the link here, just to name a few.

With the above as an overarching perspective, we are ready to jump into today’s discussion.

Research On Prompt Optimization

Sometimes you are faced with solving a problem that requires optimization. A famous example is the revered classic known as the traveling salesperson problem. A salesperson is seeking to visit clients or prospective clients in various cities. There is a distance between each city. There are costs associated with traveling from city to city. The problem consists of trying to find the optimum path for making the city visits such that you perhaps minimize the cost or some related key factor involved.

Here’s a question for you.

Can we use generative AI to solve optimization problems?

The answer is yes, you can use generative AI to solve optimization problems. A rub to this notion is that at times a conventional generative AI might do a lousy job on the optimization and not arrive at a considered optimal answer. There are chances too that generative AI will arrive at a wrong answer. You need to be especially careful when asking contemporary generative AI to solve optimization problems.

A tricky means to get conventional generative AI to cope with optimization problems involves either changing up the AI app to have additional components for that purpose or you might simply access other external apps that can do the optimization. Today’s generative AI usually readily has APIs (application programming interfaces) that allow the AI app to reach out to other apps, see my discussion at the link here. If conventional generative AI is weak at some kinds of tasks, you can often have the AI app make use of an outside app to do the additional needed work and then report back to you the results thereof.

A preference would be to get conventional generative AI to undertake optimization problems by simply using carefully composed prompts that would nudge or guide the AI app toward suitably doing so. Maybe we can tell our way into getting generative AI to adequately figure out optimization problems. This would be great because you then would not need to alter the internal capabilities of the AI app, nor would you need to become reliant on an external app that was going to do the optimization for the AI.

You would just need to use appropriate prompts to get the job done.

A recent research study by researchers at Google DeepMind sought to avidly pursue the path of prompting AI to solve optimization problems. The study is entitled “Large Language Models As Optimizers” by Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen, and was posted online on September 7, 2023.

Here are some selected excerpts in which they describe their innovative research:

The research involved making use of prompts and improving the prompts over a series of repeated tryouts, doing so in an automated way rather than having a human modify the prompts. In short, a prompt is attempted that will aim to move forward aptly on the optimization at hand. This gets the AI app to produce initial solutions. Those initial solutions are assessed and then the prompt for optimization is refined or otherwise adjusted to further improve the solutions. The hope is that this will walk the AI app step by step toward ultimately performing the optimization in a most or more satisfactory manner.

In the first part of the research study, the focus consisted of using this method for the traveling salesperson problem and also on another popular optimization problem entailing solving linear regression tasks. The second part of the research study made use of the same prompt repeating and refining technique to solve simple math or arithmetic problems from a prevalent dataset or collection known as GSM8K and another known as Big-Bench Hard (BBH) word problems collection.

I’m going to concentrate herein on the second part of the research study and in particular the GSM8K experimental portion. That being said, I certainly encourage you to ardently consider reading the entire paper as it covers fascinating and important ground on advancing efforts toward using generative AI for dealing with optimization problems. There is a lot of good stuff in there.

Any study that examines and explores generative AI is likely to pick some particular generative AI or LLM apps and use those during their study. Experimenters cannot usually try to look at all available generative AI or LLM apps since the research endeavor would be onerous and overly taxing. I mention this to alert you that whenever you hear about research studies on generative AI or LLMs, a prudent thing to discover is what particular apps were used. Keep in mind that different generative AI or LLMs will act and respond differently.

Here is what this research study chose to use:

For any AI research studies that make use of prompts, another caveat or consideration is that you should realize that the wording of prompts can be a make-or-break affair. You’ve probably observed the same when doing your prompting efforts. A word here or there in a prompt can radically impact how the AI will respond. Even simple aspects such as changing the order of the words in a prompt can have a demonstrative effect. All in all, generative AI and LLMs are typically highly sensitive to the nature of a prompt and the wording involved.

The researchers noted this same point:

Another factor entails whether the research effort opted to establish a context associated with the prompts being used. You undoubtedly know about this kind of prompting consideration. A common practice entails telling the AI app at the get-go something about what you want the AI to do or solve, see my discussion at the link here. You then proceed with prompts based on those meta-instructions or persistent considerations that you have established.

In this research study, meta-instructions were used:

Finally, in terms of getting you ready for the big reveal associated with the deep breath prompting aspects, I’d like to briefly highlight what the GSM8K collection or dataset is all about. The GSM8K is a freely available dataset consisting of essentially middle school-level math problems that were created by human problem writers. These math problems are useful for testing purposes, especially for testing AI apps. For a human solver, a typical math problem in the dataset requires somewhere between two to eight arithmetic steps to solve. The problems are often used to explore how generative AI or other AI capabilities compare to the solving of such problems.

Here's a representative example from the GSM8K. A math problem in the dataset might entail figuring out how many cookies sixteen people would consume in a week if the number of provided cookies baked was four sets of two dozen each, per week, and all of those cookies were exclusively and completely consumed by the sixteen people on a fully equal basis in a given week. Enjoy solving that one, if you wish (spoiler alert: the purported answer is six, but don’t say I told you so).

I’ve now laid out the landscape so that we can jump into the deep breath prompting reveal.

When You Need To Take A Deep Breath

Let’s home in on a chart in the study that lists test accuracies regarding the GSM8K experimental effort, indicating the highest test accuracy for each scorer-optimizer pair (I show just the prompt indicated and the corresponding accuracy score):

Inspect those prompts especially the first one listed.

It says this: “Take a deep breath and work on this problem step-by-step.”

Observe carefully that the prompt includes the wording of take a deep breath.

Some breathless pundits have opted to suggest that because this particular prompt contained the passage “take a deep breath” and because the prompt could be said to have been the winner, coming in first in the list shown, we can magnanimously declare that saying “take a deep breath” in your prompts is an extraordinary deliverance and should always be used.

Trying to be polite, let’s say that trying to reach a decisive conclusion that the “take a deep breath” is a magical or remarkable prompting element would seem to be a bit over the top. Furthermore, the study didn’t offer that as a crucial outcome or definitive finding. In that sense, others have plucked out a tiny piece and seem to overlook the bigger picture of their effort. That is unfortunate.

I would argue that before we make such a conclusion on deep breathing, additional study of the expression is needed. Research that focuses entirely or to a devoted extent on the catchphrase would be easier to reveal to what extent a generative AI or LLM will do when encountering such a passage.

Also, note that we have a commingling here in the sense that the “take a deep breath” is combined with the classic passage of thinking step-by-step. We already generally know and agree that asking generative AI work on a step-by-step basis invokes a Chain-of-Thought process and that this alone can make sizable differences in improving problem-solving (see my analysis at the link here). The study does note prior studies that used a step-by-step expression on a standalone basis, but for the reasons I note next, pundits trying to run with that should be cautious in doing so.

Let’s shift gears and go somewhat broader.

Here are twelve useful lessons and insights in general about prompting and prompt engineering that I think are worth generally pondering on these weighty matters:

I hope that those lessons will be of value in your everyday prompt engineering endeavors.

Let’s Do An Experiment, Shall We

Okay, so we have this gripping line that says this: “Take a deep breath and work on this problem step-by-step.” Divide the line into two separate concepts or portions, specifically (1) “Take a deep breath”, and (2) “Work on this problem step-by-step”.

Would the use of “Take a deep breath” on a purely standalone basis as a prompt get you any special juice in terms of what generative AI might do or generate?

I doubt it would.

Here’s why.

The “take a deep breath” expression can readily be misinterpreted by the AI. There is a chance that the generative AI will be led off on an obtuse tangent about breathing. I assume that’s not what we intend. The rest of the conversation with the AI might be tainted by an inadvertent triggering or misdirection.

Another consideration is that the “take a deep breath” is potentially entirely disregarded by the generative AI. The person entering the expression might think this is helping or causing something to occur, but the reality might be that nothing happens of any notable caliber.

I did some experimentation with ChatGPT on this. Please know that this is ad hoc rather than systematic and robust.

I outright asked ChatGPT whether “take a deep breath” would generally have any impact, and here’s the response:

That seems to provide clarity, but I acknowledge that merely because generative AI says something doesn’t mean that this is what really is going to happen under the hood. You can readily get generative AI to say one thing and do another. Thus, we should be cautious in taking the indication as ironclad in this case of having no impact. It is something we should only use as one of many clues on the matter.

I then informally provided ChatGPT with the passage of interest “Take a deep breath and work on this problem step-by-step” and asked whether the first portion about taking a deep breath would likely change or impact the problem-solving process, and here’s what I got:

Note that the AI app pointed out that deep breathing would seem to be an unrelated instruction. This suggests that adding it to the step-by-step instruction has no particular bearing or impact. Though to be clear, my qualms as above are that there is always a chance of generative AI saying one thing and doing another. Nonetheless, we do at least have an additional clue to consider. Weigh it as you wish.

Next, I opted to use the math problem about the cookies and how many cookies per person were being consumed by entering the problem into ChatGPT. The exercise consisted of varying the prompt associated with the preexisting standard wording of the math problem itself.

Here are some of many of the tryouts that I undertook:

I am sad to report that by and large for each of those prompt inclusions, the cookie's math problem was solved correctly and with an unasked-for explanation that showcases the proper logic in solving it. I made sure to start each conversation anew, aiming to avert any carryover. Time after time, the correct answer was generated.

I suppose the good news is that ChatGPT answered correctly each time. I was hoping of course to see differences. If ChatGPT got things wrong or otherwise went astray, perhaps I could attribute this to the particular prompt indication. No luck.

Can we conclude that take-a-deep-breath truly is having no impact?

I don’t think we can.

We should be mindful to not go overboard on a handful of quick ad-hoc tryouts. First, I was using a math problem that ChatGPT easily answered. There is a possibility that the take-a-deep breath might somehow make a difference on harder math problems. Or on math problems that the generative AI is unlikely to have previously been data-trained on. Likewise, maybe using problems that have nothing to do with mathematics could potentially be aided by the breathing instruction.

Unfortunately, after trying a bunch of other queries, such as telling me about the life of Abraham Lincoln, doing so with the deep breathing instruction, and then separately without it, the generated responses all seemed to be about the same. My toying with ChatGPT wasn’t getting any nifty nuances revealed.

You might be tempted to argue that deep breathing was maybe augmenting the step-by-step instruction. In other words, perhaps step-by-step as a prompt gets you 90% of the way there, while the added “take a deep breath” pushes the generative AI an additional 10% or something like that. Once again, I tried lots of variations involving including and not including the breathing instruction. Just couldn’t get anything to show up additionally via having it in the prompt.

Of course, the lack of discovering existence does not prove non-existence. That is common logic 101.

Conclusion

Let’s all take a deep breath.

Should you start adding that notable catchphrase about deep breathing to your prompts?

So far, from what seems to be the case, I would suggest that doing so won’t necessarily gain you any substantive advantage. There is also the downside that it could undercut what you are trying to do. I would generally though be amenable to agreeing that it probably won’t usually hurt or cause a problem when solving a problem.

If you like saying it, go for it.

Are there circumstances in which bringing up the taking of a deep breath might be especially worthwhile?

Sure, such as when I asked about the life of Lincoln. I did get an interesting answer that began with the AI app saying that if you take a deep breath then you can see Lincoln’s life in a manner that otherwise you might not have. That was a clever echoing of my prompt indication. Turns out, the essay was pretty much the same as the other essays on Lincoln that I asked for in which I had not mentioned the breathing consideration. The breathing aspect was carried over as a flavor or style, possibly mimicking my having used it. Regrettably, otherwise, the essay was about the same in size, quality, etc.

I suppose we all could from time to time benefit from the use of the “take a deep breath” catchphrase in our daily lives. The magical powers of doing so in generative AI seem limited. If I discover something otherwise, I’ll be sure to let you all know.

A final comment for now. There are indubitably magical powers of taking a deep breath when you compose a prompt and try to anticipate what the prompt will generate.

That’s a highly recommended deep breath to take.