NEOACM REMOTE OPEN DISCUSSION: From Prompt Engineering to Prompt Science

We will have an open discussion about this article:

From Prompt Engineering to Prompt Science with Humans in the Loop
Can prompt engineering be made more reliable, verifiable, and generalizable?

that appeared in the May Issue of COMMUNICATIONS OF THE ACM. Here is a link to the article:

https://tinyurl.com/mrsw2kjx

Here are some snippets from the article:

“In recent years, as the sophistication and capabilities of large language models (LLMs) have grown, so have the tasks for which they’re applicable, going beyond information extraction and synthesis to include analysis, content creation, and reasoning. Unsurprisingly, many researchers find them useful for research tasks, such as identifying relevant papers, synthesizing literature reviews, writing proposals, and analyzing data. They have also been found effective for investigative tasks, such as drug discovery. There is growing concern, however, that a large portion of this success hinges on prompt engineering, which is often an ad-hoc method to revise prompts being fed into an LLM to achieve a desired response or analysis...

In most cases, overreliance on prompt engineering can lead to unexplainable, unverifiable, and less generalizable outcomes,lacking a scientific approach.

We consider a scientific approach to be the one that is well-documented and transparent, verifiable, repeatable, and free of individual biases and subjectivity. Most prompt-engineering techniques violate at least one of these principles. While such ad-hocness can be bad in most situations, it is especially problematic when it comes to scientific research, which demands a certain level of rigor. Thus, if we want to continue harnessing the power of LLMs, we need to do so in a responsible manner and with enough scientific rigor.”

In this open discussion, we will open the conversation with what are “PROMPTS.” They are natural language requests. What is the history of programming computers with “natural language interface”?

Questions:

1. Can “prompting” be transformed into something verifiable using replicable methods utilizing qualitative coding?

2. Will this “transformation” evolve into duplicating SQL or declarative programming?

3. Will this new prompting science require programmers? The purpose was not to use a programming paradigm.

We hope you will participate in this discussion!! Please RSVP, we will email zoom credentials.

Thank You
Tracey Hughes (NEOACM Secretary)