small experiment on handpicked samples

ideas_practice/code/.gitignore

 .ipynm_checkpoints/*
 .ipynb_checkpoints/*
+*.env

ideas_practice/code/bioasq_yesno_handpicked_experiment.csv

+id,question,ground_truth,prompt,prompting_strategy,answer,date_time,model
+1,Does metformin interfere thyroxine absorption?,no,,baseline,No,30.05.2024 from 18:00 to 18:30,GPT-3.5
+1,Does metformin interfere thyroxine absorption?,no,,aionr,No,30.05.2024 from 19:55 to 20:05,GPT-3.5
+1,Does metformin interfere thyroxine absorption?,no,,aionlr,No,30.05.2024 from 20:10 to 00:00,GPT-3.5
+1,Does metformin interfere thyroxine absorption?,no,,cot,,,
+4,Are there any specific antidotes for rivaroxaban?,no,,baseline,No,30.05.2024 from 18:00 to 18:30,GPT-3.5
+4,Are there any specific antidotes for rivaroxaban?,no,,aionr,No,30.05.2024 from 19:55 to 20:05,GPT-3.5
+4,Are there any specific antidotes for rivaroxaban?,no,,aionlr,No,30.05.2024 from 20:10 to 00:00,GPT-3.5
+4,Are there any specific antidotes for rivaroxaban?,no,,cot,,,
+7,Is it safe to take isotretinoin during pregnancy?,no,,baseline,No,30.05.2024 from 18:00 to 18:30,GPT-3.5
+7,Is it safe to take isotretinoin during pregnancy?,no,,aionr,No,30.05.2024 from 19:55 to 20:05,GPT-3.5
+7,Is it safe to take isotretinoin during pregnancy?,no,,aionlr,No,30.05.2024 from 20:10 to 00:00,GPT-3.5
+7,Is it safe to take isotretinoin during pregnancy?,no,,cot,,,
+9,Have thyronamines  effects on fat tissue?,no,,baseline,Yes,30.05.2024 from 18:00 to 18:30,GPT-3.5
+9,Have thyronamines  effects on fat tissue?,no,,aionr,Yes,30.05.2024 from 19:55 to 20:05,GPT-3.5
+9,Have thyronamines  effects on fat tissue?,no,,aionlr,Yes,30.05.2024 from 20:10 to 00:00,GPT-3.5
+9,Have thyronamines  effects on fat tissue?,no,,cot,,,
+11,Is there an association between borna virus and brain tumor?,no,,baseline,No,30.05.2024 from 18:00 to 18:30,GPT-3.5
+11,Is there an association between borna virus and brain tumor?,no,,aionr,No,30.05.2024 from 19:55 to 20:05,GPT-3.5
+11,Is there an association between borna virus and brain tumor?,no,,aionlr,No,30.05.2024 from 20:10 to 00:00,GPT-3.5
+11,Is there an association between borna virus and brain tumor?,no,,cot,,,
+13,Is endostatin a proangiogenic factor?,no,,baseline,No,30.05.2024 from 18:00 to 18:30,GPT-3.5
+13,Is endostatin a proangiogenic factor?,no,,aionr,No,30.05.2024 from 19:55 to 20:05,GPT-3.5
+13,Is endostatin a proangiogenic factor?,no,,aionlr,No,30.05.2024 from 20:10 to 00:00,GPT-3.5
+13,Is endostatin a proangiogenic factor?,no,,cot,,,
+20,Are chromomethylases present in animal genomes?,no,,baseline,No,30.05.2024 from 18:00 to 18:30,GPT-3.5
+20,Are chromomethylases present in animal genomes?,no,,aionr,Yes,30.05.2024 from 19:55 to 20:05,GPT-3.5
+20,Are chromomethylases present in animal genomes?,no,,aionlr,No,30.05.2024 from 20:10 to 00:00,GPT-3.5
+20,Are chromomethylases present in animal genomes?,no,,cot,,,
+33,Is it safe to use Abatacept during pregnancy?,no,,baseline,No,30.05.2024 from 18:00 to 18:30,GPT-3.5
+33,Is it safe to use Abatacept during pregnancy?,no,,aionr,No,30.05.2024 from 19:55 to 20:05,GPT-3.5
+33,Is it safe to use Abatacept during pregnancy?,no,,aionlr,No,30.05.2024 from 20:10 to 00:00,GPT-3.5
+33,Is it safe to use Abatacept during pregnancy?,no,,cot,,,
+34,Are there any specific antidotes for dabigatran?,no,,baseline,Yes,30.05.2024 from 18:00 to 18:30,GPT-3.5
+34,Are there any specific antidotes for dabigatran?,no,,aionr,No,30.05.2024 from 19:55 to 20:05,GPT-3.5
+34,Are there any specific antidotes for dabigatran?,no,,aionlr,No,30.05.2024 from 20:10 to 00:00,GPT-3.5
+34,Are there any specific antidotes for dabigatran?,no,,cot,,,
+36,Is armodafinil used for treatment of insomnia?,no,,baseline,No,30.05.2024 from 18:00 to 18:30,GPT-3.5
+36,Is armodafinil used for treatment of insomnia?,no,,aionr,Yes,30.05.2024 from 19:55 to 20:05,GPT-3.5
+36,Is armodafinil used for treatment of insomnia?,no,,aionlr,No,30.05.2024 from 20:10 to 00:00,GPT-3.5
+36,Is armodafinil used for treatment of insomnia?,no,,cot,,,
+355,Is the protein Papilin secreted?,yes,,baseline,Yes,30.05.2024 from 18:00 to 18:30,GPT-3.5
+355,Is the protein Papilin secreted?,yes,,aionr,Yes,30.05.2024 from 19:55 to 20:05,GPT-3.5
+355,Is the protein Papilin secreted?,yes,,aionlr,Yes,30.05.2024 from 20:10 to 00:00,GPT-3.5
+355,Is the protein Papilin secreted?,yes,,cot,,,
+360,Are transcription and splicing connected?,yes,,baseline,Yes,30.05.2024 from 18:00 to 18:30,GPT-3.5
+360,Are transcription and splicing connected?,yes,,aionr,No,30.05.2024 from 19:55 to 20:05,GPT-3.5
+360,Are transcription and splicing connected?,yes,,aionlr,Yes,30.05.2024 from 20:10 to 00:00,GPT-3.5
+360,Are transcription and splicing connected?,yes,,cot,,,
+361,Is Alu hypomethylation associated with breast cancer?,yes,,baseline,Yes,30.05.2024 from 18:00 to 18:30,GPT-3.5
+361,Is Alu hypomethylation associated with breast cancer?,yes,,aionr,Yes,30.05.2024 from 19:55 to 20:05,GPT-3.5
+361,Is Alu hypomethylation associated with breast cancer?,yes,,aionlr,Yes,30.05.2024 from 20:10 to 00:00,GPT-3.5
+361,Is Alu hypomethylation associated with breast cancer?,yes,,cot,,,
+363,Do mutations of AKT1 occur in meningiomas?,yes,,baseline,Yes,30.05.2024 from 18:00 to 18:30,GPT-3.5
+363,Do mutations of AKT1 occur in meningiomas?,yes,,aionr,Yes,30.05.2024 from 19:55 to 20:05,GPT-3.5
+363,Do mutations of AKT1 occur in meningiomas?,yes,,aionlr,Yes,30.05.2024 from 20:10 to 00:00,GPT-3.5
+363,Do mutations of AKT1 occur in meningiomas?,yes,,cot,,,
+368,Is Weaver syndrome similar to Sotos?,yes,,baseline,No,30.05.2024 from 18:00 to 18:30,GPT-3.5
+368,Is Weaver syndrome similar to Sotos?,yes,,aionr,No,30.05.2024 from 19:55 to 20:05,GPT-3.5
+368,Is Weaver syndrome similar to Sotos?,yes,,aionlr,No,30.05.2024 from 20:10 to 00:00,GPT-3.5
+368,Is Weaver syndrome similar to Sotos?,yes,,cot,,,
+379,Is valproic acid effective for glioblastoma treatment?,yes,,baseline,No,30.05.2024 from 18:00 to 18:30,GPT-3.5
+379,Is valproic acid effective for glioblastoma treatment?,yes,,aionr,No,30.05.2024 from 19:55 to 20:05,GPT-3.5
+379,Is valproic acid effective for glioblastoma treatment?,yes,,aionlr,No,30.05.2024 from 20:10 to 00:00,GPT-3.5
+379,Is valproic acid effective for glioblastoma treatment?,yes,,cot,,,
+380,Can Levoxyl (levothyroxine sodium) cause insomnia?,yes,,baseline,Yes,30.05.2024 from 18:00 to 18:30,GPT-3.5
+380,Can Levoxyl (levothyroxine sodium) cause insomnia?,yes,,aionr,Yes,30.05.2024 from 19:55 to 20:05,GPT-3.5
+380,Can Levoxyl (levothyroxine sodium) cause insomnia?,yes,,aionlr,Yes,30.05.2024 from 20:10 to 00:00,GPT-3.5
+380,Can Levoxyl (levothyroxine sodium) cause insomnia?,yes,,cot,,,
+427,Is ospemifene effective for treatment of dyspareunia?,yes,,baseline,Yes,30.05.2024 from 18:00 to 18:30,GPT-3.5
+427,Is ospemifene effective for treatment of dyspareunia?,yes,,aionr,Yes,30.05.2024 from 19:55 to 20:05,GPT-3.5
+427,Is ospemifene effective for treatment of dyspareunia?,yes,,aionlr,Yes,30.05.2024 from 20:10 to 00:00,GPT-3.5
+427,Is ospemifene effective for treatment of dyspareunia?,yes,,cot,,,
+430,Is RET the major gene involved in Hirschsprung disease?,yes,,baseline,Yes,30.05.2024 from 18:00 to 18:30,GPT-3.5
+430,Is RET the major gene involved in Hirschsprung disease?,yes,,aionr,Yes,30.05.2024 from 19:55 to 20:05,GPT-3.5
+430,Is RET the major gene involved in Hirschsprung disease?,yes,,aionlr,Yes,30.05.2024 from 20:10 to 00:00,GPT-3.5
+430,Is RET the major gene involved in Hirschsprung disease?,yes,,cot,,,
+441,Can sorafenib activate AMPK?,yes,,baseline,No,30.05.2024 from 18:00 to 18:30,GPT-3.5
+441,Can sorafenib activate AMPK?,yes,,aionr,No,30.05.2024 from 19:55 to 20:05,GPT-3.5
+441,Can sorafenib activate AMPK?,yes,,aionlr,No,30.05.2024 from 20:10 to 00:00,GPT-3.5
+441,Can sorafenib activate AMPK?,yes,,cot,,,

ideas_practice/code/prompting_strategies.md

+This file describes the different strategies used for the prompting process and how the prompts are created. We considered the guidlines offered by OpenAI https://platform.openai.com/docs/guides/prompt-engineering/
+
+## Baseline (baseline):
+The questions will be given to the model as is without any additional context. 
+Prompts have the following format:
+"""
+You are a helpful medical expert who answers with either yes or no to questions.
+
+Question: <question> 
+"""
+
+## All in One None Resolve (aionr):
+The questions will be given to the model as is with additional triples in their raw format.
+Prompts have the following format:
+"""
+You are a helpful medical expert who answers with either yes or no. Consider the information in triple quotes as factual knowledge.
+
+"""<s1> has relationshsip <r1> to <o1>"""
+"""<s2> has relationshsip <r2> to <o2>"""
+
+Question: <question>
+"""
+
+## All in One NL Resolve (aionlr):
+The questions will be given to the model as is with additional triples in the form of natural sentences.
+Prompts have the following format:
+"""
+You are a helpful medical expert who answers with either yes or no. Consider the information in triple quotes as factual knowledge.
+
+"""<NL sentence for s1r1o1>"""
+"""<NL sentence for s2r2o2>"""
+
+Question: <question>
+"""
+
+## Chain of Thought (cot):
+The questions will be given to the model as is. After the answer the model will be iteratively asked to reevaluate their answer given a triple in NL sentence form.
+"""
+You are a helpful medical expert who answers with either yes or no. Consider the information in triple quotes as factual knowledge.
+
+Question: <question>
+"""
+
+"""
+Given our previous conversation and the following information in triple quotes reevaluate your answer.
+"""<NL sentence for s1r1o1>"""
+"""
+
+"""
+Given our previous conversation and the following information in triple quotes reevaluate your answer.
+"""<NL sentence for s2r2o2>"""
+"""