Dataset
First 50 rows of train
| datasetstring | agent_ref | responses_create_params | expected_answerfloat64 | answer_typestring | output_regexstring | propertystring | methodstring | question_licensestring | metadata | question_familystring | matchJson |
|---|---|---|---|---|---|---|---|---|---|---|---|
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Report the QED of this molecule.\n\nCOc1cc2c(cc1O)CN(C(=O)C(C)C)CC2\n\nProvide your final answer as '<Answer: floating_point_number>', where floating_point_number is the answer. The final result should be a single floating point number."}]} | 0.871091 | float | <?Answer:\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"75b5d882057e192ad8c83c1b752fcbc3","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL488878"}]} | tier2_rdkit | |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Compute the QED score for this compound.\n\nCCOc1ccc(OCC)c(NC(=O)C2CCN(S(=O)(=O)c3cccc4nonc34)CC2)c1\n\nConclude your response with \"**Final answer is: floating_point_number**\", where floating_point_number is the answer. The final result shou… | 0.5282712 | float | \*{0,2}Final answer is:\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) |
| direct |
| CC-BY-SA-4.0 |
| {"uuid":"e2e6fff1f389101b8ee31f2a04e4530f","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL1350496"}]} |
| tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Compute the QED score of the following SMILES.\n\nCC(C)c1cccc(C(C)C)c1NC(=O)NCC1(c2cccs2)CCCC1\n\nYour final answer must be enclosed in double parentheses. Do not modify the required answer format with any markdown, code fences, quotes, comment… | 0.57648504 | float | \(\(\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"0fd05ee7e9f9d976cebbc8376a368b0a","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL430887"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"What is the predicted quantitative estimate of drug-likeness (QED) of the compound below? Tell me as a decimal number.\n\nCC(OC1=CC(=O)CCC1)C(=O)c1ccccc1\n\nPlace the final answer within \\boxed{}. Do not modify the required answer format with … | 0.7647432 | float | \\boxed\{\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"db0d24cd3cb0ac16da6c2e4504918418","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL1566727"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Report the quantitative estimate of drug-likeness (QED) of the compound below.\n\nCOc1ccc(C(=O)OCC(=O)c2ccc[nH]2)cc1\n\nEnd with \"ANSWER IS floating_point_number\" on a new line, where floating_point_number is the final answer. The final resul… | 0.6588716 | float | ANSWER IS\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"4cbfe38a3ba085de654ddb1aa0775398","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL1555413"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"What is the predicted QED score for this compound?\n\nC[C@H]1[C@@H](OCCNCCCn2ccnc2)O[C@@H]2O[C@@]3(C)CC[C@H]4[C@H](C)CC[C@@H]1[C@@]24OO3\n\nEnd with the answer wrapped like this: <<floating_point_number>>. The final result should be a single fl… | 0.4966941 | float | <<\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"7f34b6726cdc69c0dd6aaafabb7f6094","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL2079040"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Compute the quantitative estimate of drug-likeness (QED) of this molecule.\n\nO=[N+]([O-])c1ccc(N2CCN(C(=S)N/N=C\\c3ccc4ccccc4n3)CC2)cc1\n\nEnd your response with [Answer: floating_point_number], where floating_point_number is the final answer.… | 0.300196 | float | \[?Answer:\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"ddab1b93eea1a369c5fd40d4086bae2f","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL4448539"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Determine the quantitative estimate of drug-likeness (QED) for the following molecule. Give a numeric value.\n\nO=C(N/N=C/c1ccc(Cl)cc1Cl)c1cnccn1\n\nConclude your response with: Correct Answer >> floating_point_number. The final result should b… | 0.69865185 | float | Correct Answer >>\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"ea74aa4412a07674964f56b9632d46c8","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL2311868"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"What is the predicted QED of the compound below? Tell me as a decimal number.\n\nCn1cc(Nc2nccc(-c3ccc4c(c3)CCN(CC(C)(C)O)CC4NC(=O)c3nnc(C(C)(C)C)o3)n2)cn1\n\nPlace your final answer within double angle brackets at the end of your response: <<fl… | 0.30772227 | float | <<\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"c35442af3bc6cb06a597a367115406f6","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL5959304"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Report the quantitative estimate of drug-likeness (QED) for the molecule shown. Tell me as a decimal number.\n\nCCOC(=O)Cc1nc(-c2ccc(Cl)cc2)oc1-c1sccc1Br\n\nPlace your final answer within double angle brackets at the end of your response: <<flo… | 0.49890155 | float | <<\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"d528ecc6302e4803671e9ed83d3918a6","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL114476"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Calculate the QED of the compound below. Tell me as a decimal number.\n\nCc1n[nH]c(C)c1S(=O)(=O)N1C[C@H]2CC[C@@H](C1)[C@H]2Oc1cnccn1\n\nProvide the final answer at the end of your response in this exact format: Final Answer -> floating_point_nu… | 0.87842745 | float | Final Answer\s*->\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"a701a8dadbd53e4409e0a9c2ff2546ea","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL4850107"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"What is the QED for this compound?\n\nC=C(C)[C@H]1CCc2c(c(C)cc(O)c(=O)c2O)C1\n\nEnd your response with the answer in the format: Answer Value: {floating_point_number}. The final result should be a single floating point number."}]} | 0.7482326 | float | Answer Value:\s*\*{0,2}\{?\s*([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"23c0af667177b5ac948bf909e78f6e11","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL1802031"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"What is the predicted quantitative estimate of drug-likeness (QED) of the following SMILES?\n\nCc1nc(C(=O)N2Cc3cc4c(cc3C[C@H]2C(=O)N[C@@H](Cc2ccc(-c3ccnc(C)c3C)cc2)C(=O)O)OC[C@H](c2ccc(OCc3cccc(Cl)c3Cl)cc2)O4)c(C)o1\n\nConclude your response wi… | 0.12934563 | float | \*{0,2}Final answer is:\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"de25372a82f955ee16067346ff353a9f","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL3950460"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Report the quantitative estimate of drug-likeness (QED) for this compound.\n\nCOc1ccc(NC(=O)C2CC(=O)N(c3cccc(Cl)c3)C(=O)N2NC(=O)c2ccncc2)cc1\n\nProvide the answer at the end, enclosed in single angle brackets: <floating_point_number>. The final… | 0.543427 | float | <\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?)\*{0,2}\s*> | qed | direct | CC-BY-SA-4.0 | {"uuid":"e03161402556e0948676c179a33d14ad","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL1392172"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Compute the QED for the following molecule.\n\nO=S(=O)(c1cccc(F)c1)N1CCN(c2nc(-c3cccs3)nc3ccccc23)CC1\n\nYour final answer must be enclosed in double curly braces. Do not modify the required answer format with any markdown, code fences, quotes,… | 0.46623018 | float | \{\{\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"ecaa5a08d3a3360995bc184cf2695f94","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL1515820"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Determine the quantitative estimate of drug-likeness (QED) for the molecule shown.\n\nCOc1ccc2cc(-c3cccnc3)ccc2c1Cl\n\nThe final answer must be placed in a box using the \\boxed{} format. Do not modify the required answer format with any markdo… | 0.6801157 | float | \\boxed\{\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"d2c7352f92f8af96922a7becf09c9173","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL197576"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Compute the quantitative estimate of drug-likeness (QED) of the compound below. Give a numeric value.\n\nCNC(=O)c1cnncc1NS(=O)(=O)c1ccc(Oc2ccccc2Cl)cc1\n\nAt the conclusion of your response, provide the final answer in the format: ## Answer: fl… | 0.63605404 | float | #{0,2}\s*Answer:\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"a82182a823987e56a264b70e3a7cf6eb","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL5606179"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Report the QED for the molecule shown.\n\nCc1cnc(Nc2ccc(N3CCCCC3)cc2)nc1Nc1nc2ccccc2s1\n\nEnd your response with [Answer: floating_point_number], where floating_point_number is the final answer. The final result should be a single floating poin… | 0.42394137 | float | \[?Answer:\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"f4704f0ad994f4285791a325f348808f","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL4564823"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Calculate the QED score for the following molecule. Tell me as a decimal number.\n\nCC(=O)N1CCC(C#N)(c2cc(Nc3cc(C(F)(F)F)ccn3)nc(N3CCC(F)(F)C3)n2)CC1\n\nPlace your final answer within double angle brackets at the end of your response: <<floatin… | 0.6431449 | float | <<\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"4b78d396c5b5815e439cccb2c31d4db7","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL5882316"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"What is the predicted quantitative estimate of drug-likeness (QED) of the compound below? Give a numeric value.\n\nCN(CCOc1ccc(CC(Nc2ccccc2S(=O)(=O)c2ccccc2)C(=O)O)cc1)c1nc2ccccc2o1\n\nConclude your response with \"**Final answer is: floating_p… | 0.20583884 | float | \*{0,2}Final answer is:\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"46f7ec2706f818bbd6c6c4bb20807ca1","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL146123"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"What is the predicted quantitative estimate of drug-likeness (QED) for the following molecule? Tell me as a decimal number.\n\nCc1ccc(Cl)cc1-n1c(CCC(=O)O)ccc1-c1cccs1\n\nEnd your response with Final Answer: ||floating_point_number||, where floa… | 0.69145 | float | Final Answer:\s*\*{0,2}\|{0,2}\s*([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"2a8f0c0a1e8b96ea19d91e11b63ed3ec","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL1322610"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"What is the predicted quantitative estimate of drug-likeness (QED) of the compound below? Give a numeric value.\n\nCN(CCCCCNC(=O)/C=N/O)Cc1ccccc1\n\nEnd your response with [Answer: floating_point_number], where floating_point_number is the fina… | 0.3133049 | float | \[?Answer:\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"74b3645fe756c0aa3c55923b5b67afac","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL3209103"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Calculate the QED score of the following SMILES.\n\nC#CCNC/C=C/c1ccc(O)c(OC)c1\n\nProvide the answer at the end, enclosed in single angle brackets: <floating_point_number>. The final result should be a single floating point number."}]} | 0.5816426 | float | <\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?)\*{0,2}\s*> | qed | direct | CC-BY-SA-4.0 | {"uuid":"ed54b8969b2a35bfa963b28a5954c5cb","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL5077905"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Determine the quantitative estimate of drug-likeness (QED) for this compound. Tell me as a decimal number.\n\nClc1ccc2c(N3CCN(Cc4ccccc4)CC3)ccnc2c1\n\nConclude your response with: Correct Answer >> floating_point_number. The final result should… | 0.7133422 | float | Correct Answer >>\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"8248dede0cfca6d225e4b7019ed877a4","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL1256765"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Determine the QED of this molecule. Give a numeric value.\n\nCc1nc(O)c2c(cc(-c3ccc(F)nc3F)c3nc(Nc4c(Cl)cccc4Cl)n(C)c32)c1C\n\nPlace your final answer at the end of your response, enclosed within XML-style tags as follows: `<final_answer>your an… | 0.2656525 | float | <final_answer>\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"ca90fac3e59672d940be68bcf1fa46e9","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL26625"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"What is the predicted QED of this molecule?\n\nCOc1cc2c3c(n(CCCCCO)c(=O)c2cc1OC)-c1cc2c(cc1C3=O)OCO2\n\nYour final answer should be on a new line as 'Final value is: floating_point_number'. The final result should be a single floating point num… | 0.44390553 | float | Final value is:\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"77afa6c3a3c491dcabd534c79e1f4b9b","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL141212"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"What is the QED of this molecule?\n\nCC(C)(C)OC(=O)N[C@H](CC(N)=O)C(=O)O\n\nAt the end of your response, state the answer in the following format: 'Correct answer is floating_point_number'. The final result should be a single floating point num… | 0.6218489 | float | Correct answer is\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"47ab947293caac299e94e957397c007a","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL1222169"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"What is the predicted QED score of the following SMILES?\n\nCC(C)[C@@H](NS(=O)(=O)c1ccc(NC(=O)c2ccc(O[11CH3])cc2)cc1)C(=O)O\n\nFor the final answer, put 'The answer is: |floating_point_number|' at the end, where floating_point_number is the ans… | 0.6196402 | float | The answer is:\s*\*{0,2}\|?\s*([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"81bcb2cdc9c30a2cae876c5211636720","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL2311189"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Calculate the QED score for this compound.\n\nC[C@@]1(c2cc(CNC34CC(C3)C4)c(F)cc2F)CCSC(N)=N1\n\nEnd your response with Final Answer: ||floating_point_number||, where floating_point_number is the final answer. Do not modify the required answer f… | 0.88645 | float | Final Answer:\s*\*{0,2}\|{0,2}\s*([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"e5470866447b4f2b8876f7b655b1930c","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL4102593"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Compute the quantitative estimate of drug-likeness (QED) for the molecule shown. Tell me as a decimal number.\n\nCc1nc(C[P+](c2cccs2)(c2cccs2)c2cccs2)c(C[P+](c2cccs2)(c2cccs2)c2cccs2)c2c1OC(C)(C)OC2\n\nProvide the answer at the end, enclosed in… | 0.13021517 | float | <\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?)\*{0,2}\s*> | qed | direct | CC-BY-SA-4.0 | {"uuid":"c5814da53cd11fbcc045281f34128370","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL3140005"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"What is the quantitative estimate of drug-likeness (QED) for the molecule shown? Give a numeric value.\n\nCN[C@@H](C)C(=O)Nc1ccc2c(n1)CN(C(C)CC(C)(C)C)C(=O)N2c1cccc(Cl)c1\n\nAt the end of your response, provide the final answer in the format: F… | 0.6260607 | float | Final answer:\s*\*{0,2}\(?\s*([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"7a885ff438cd7eedcdec74b4ce7f9c44","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL3896283"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"What is the quantitative estimate of drug-likeness (QED) of the following SMILES?\n\nFC(F)Oc1cc(CNc2ncnc3[nH]ncc23)ccn1\n\nProvide the answer at the end, enclosed in single angle brackets: <floating_point_number>. The final result should be a s… | 0.7470157 | float | <\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?)\*{0,2}\s*> | qed | direct | CC-BY-SA-4.0 | {"uuid":"780219fb748ab750545753c8a974c2ca","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL5309218"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Report the quantitative estimate of drug-likeness (QED) of this molecule. Tell me as a decimal number.\n\nCc1cccc(OCC2CCN(c3ccc(-c4nc(C(F)(F)F)cn4COCC[Si](C)(C)C)cn3)CC2)c1\n\nAt the end of your response, provide the final answer in the format:… | 0.20497528 | float | Final answer:\s*\*{0,2}\(?\s*([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"035598a2d8694a0f7d19c3085b8fb6e2","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL5923692"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"What is the QED score for the following molecule?\n\nCCN(CC)CCCNc1nnc(C)c(O)n1\n\nProvide your final answer at the end of the response as: Answer: *floating_point_number*, where floating_point_number is the answer. The final result should be a … | 0.6922475 | float | Answer:\s*\*{0,2}\*?\s*([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"f5203e9f84135bdbcd2c979c51d96e42","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL1408895"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"What is the predicted quantitative estimate of drug-likeness (QED) for the following molecule?\n\nCOc1c(N2CCN(C(c3ccccc3)c3ccc(Cl)cc3)CC2)c(F)c([N+](=O)[O-])c2c(=O)c(C(=O)O)cn(C3CC3)c12\n\nProvide your final answer as '<Answer: floating_point_n… | 0.20176311 | float | <?Answer:\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"f7d1cfcb75454c30efa387bc4112c101","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL260035"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Calculate the quantitative estimate of drug-likeness (QED) for the following molecule.\n\nC[C@@H]1CN(c2ccc(F)cc2C(F)(F)F)CCN1S(=O)(=O)c1ccc(C#N)cc1\n\nEnd your response with the answer in the format: Answer Value: {floating_point_number}. The f… | 0.7023525 | float | Answer Value:\s*\*{0,2}\{?\s*([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"2724d333b6222c8b3cc708c09bc48e5b","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL551186"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"What is the QED of this molecule?\n\nCc1cc(Cl)c(-c2cn(C)c(C)n2)cc1NC(=O)c1ccc(OCc2ccccn2)cc1\n\nProvide the final answer at the end of your response in this exact format: Final Answer -> floating_point_number. Do not modify the required answer … | 0.41922286 | float | Final Answer\s*->\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"25cacc858dc2dfb02f986c4d87284946","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL4636369"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Calculate the QED of the following SMILES.\n\nCc1ccc(-c2cc(C(=O)N[C@H](C)CN3CCS(=O)(=O)CC3)cc(-c3scnc3C)c2)cc1\n\nThe final answer must be placed in a box using the \\boxed{} format. Do not modify the required answer format with any markdown, c… | 0.57449865 | float | \\boxed\{\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"e806186a2b2fc7f7f3bd5970db6c7170","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL3717684"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Compute the quantitative estimate of drug-likeness (QED) for the following molecule.\n\nCOc1cc2c(cc1-c1cn([C@H]3CS(=O)(=O)C[C@@H]3O)nn1)CCC2\n\nYour final answer must be enclosed in double curly braces. Do not modify the required answer format … | 0.8796487 | float | \{\{\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"24a94369c20ce183468ea22f002cf662","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL4924221"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Determine the QED of the following SMILES. Give a numeric value.\n\nCCCn1c(=O)c(C(=O)Nc2cc(C)ccn2)c(O)c2ccccc21\n\nEnd your response with Final Answer: ||floating_point_number||, where floating_point_number is the final answer. Do not modify th… | 0.76613176 | float | Final Answer:\s*\*{0,2}\|{0,2}\s*([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"3f973974638582c29b1ba443acefd11b","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL1453138"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Calculate the QED score for the following molecule. Tell me as a decimal number.\n\nCc1cccnc1[C@@H]1CCC[C@H](c2nc3ccccn3c2CCC#N)N1\n\nPlace the final answer within \\boxed{}. Do not modify the required answer format with any markdown, code fenc… | 0.77452976 | float | \\boxed\{\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"bc21a5b9e7c41abf3b9769261362a793","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL5783873"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Report the QED of this molecule. Give a numeric value.\n\nCC1(C)[C@H]2CC[C@]1(c1ccnc(-n3cnc(CCC(N)=O)n3)n1)c1nnc(-c3c(F)cccc3F)cc12\n\nConclude with (Answer: floating_point_number), where floating_point_number is the final answer. The final res… | 0.4282357 | float | \(?Answer:\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"71e7a44cd8038ec990751f104877c1e6","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL5921647"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"What is the predicted quantitative estimate of drug-likeness (QED) of the following SMILES? Tell me as a decimal number.\n\nO=C(NC1c2ccc(Cl)cc2Oc2cccc(Cl)c21)c1ccc[nH]c1=O\n\nYour final answer must be enclosed in double parentheses. Do not modi… | 0.68867445 | float | \(\(\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"fa8267fe4f4e211b995aeba6134041f4","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL5078221"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Report the quantitative estimate of drug-likeness (QED) for the molecule shown. Tell me as a decimal number.\n\nNc1ncc(-c2ccc(S(=O)(=O)N[C@@H]3CCNC3)c(S(N)(=O)=O)c2-c2nn[nH]n2)s1\n\nProvide your final answer at the end of the response as: Answe… | 0.29129508 | float | Answer:\s*\*{0,2}\*?\s*([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"9227f6f2bc70261ba87067a8ff426782","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL5954264"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Calculate the quantitative estimate of drug-likeness (QED) for this compound. Tell me as a decimal number.\n\nO=C(Nc1nnc(SCC(=O)N2CCOCC2)s1)C1CC(=O)N(c2ccc(F)cc2)C1\n\nPlace the final answer within \\boxed{}. Do not modify the required answer f… | 0.5106043 | float | \\boxed\{\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"e2f1649841800016047055598ab5c681","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL1370935"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Report the QED score of this molecule. Tell me as a decimal number.\n\nCS(=O)(=O)NC1CC2(CCN(CCF)CC2)Oc2ccccc21\n\nProvide the final answer at the end of your response in this exact format: Final Answer -> floating_point_number. Do not modify th… | 0.90741163 | float | Final Answer\s*->\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"48f155b92740accb67f47acf601ac6a6","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL3557108"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"What is the QED score for the molecule shown? Give a numeric value.\n\nCOc1ccc(CNC(=O)c2ccccc2)cc1OC\n\nEnd your response with Final Answer: ||floating_point_number||, where floating_point_number is the final answer. Do not modify the required … | 0.9090447 | float | Final Answer:\s*\*{0,2}\|{0,2}\s*([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"b062fbbd95a5b6f6ad820be172f42c29","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL2268813"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"Compute the quantitative estimate of drug-likeness (QED) of the following SMILES. Tell me as a decimal number.\n\nC[S+]([O-])c1nnc(-c2ccc(OCc3cccc(Cl)c3)c(Cl)c2)o1\n\nEnd your response with the final answer on a new line in the format: Response… | 0.61251444 | float | Response:\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"32cb8455884aa50543efdbb34aea5cbb","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL482140"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"What is the predicted quantitative estimate of drug-likeness (QED) of this molecule? Give a numeric value.\n\nCOC[C@@H](c1ccc(C(F)(F)F)cc1)N1CCN(C2C[C@@H]3CN(C(=O)c4c(C)cc(C#N)nc4C)C[C@@H]3C2)C[C@@H]1C\n\nEnd your response with [Answer: floatin… | 0.4963377 | float | \[?Answer:\s*\*{0,2}([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"7cb7656ee314bf4b71c3b1efc5e00362","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL1077352"}]} | tier2_rdkit |
| litmus_bench_v0.2 | {"name":"litmus_agent_agent"} | {"input":[{"role":"user","content":"What is the predicted quantitative estimate of drug-likeness (QED) of the compound below? Give a numeric value.\n\nCC(=O)N[C@H]1[C@H]([C@H](O)[C@H](O)CO)C[C@@](O[C@@H]2[C@H](O)[C@H](O[C@H]3[C@H](O[C@@H]4O[C@@H](C)[C@@H](O)[C@@H](O)[C@@H]4O)[C@H… | 0.07735367 | float | Answer:\s*\*{0,2}\*?\s*([-+]?\d*\.?\d+(?:[eE][-+]?\d+)?) | qed | direct | CC-BY-SA-4.0 | {"uuid":"cd5e165a9a21f472d49445d8bfe1538d","sources":[{"dataset":"chembl","dataset_version":"36","row_id":"CHEMBL3215473"}]} | tier2_rdkit |
