First 50 rows of validation
| idstring | kindstring | optionsList | targetList | statestring | questionstring | domainstring | familystring | sourcestring |
|---|---|---|---|---|---|---|---|---|
| v3_55fa8d2318e54bce_s | score | ["0","1","2","3","4","5"] | [0,0.08,0.26,0.49,0.16,0.01] | A geiger counter shows quantization at 1/3 LSB; additionally tracks the reference within 0.05%. Context: during the shock test, immediately after go-live. Reported by the supervisor. | Rate calibration urgency for this scenario on a 0-5 scale. | instrumentation | physics |
| v3_ee950df8c330cd92_c | choice | ["issue_warning","renegotiate","dual_source","maintain"] | [0.6,0.07,0.32,0.01] | A distribution center shows a count variance of 2.3%; additionally reports stock at 8% of safety level. Context: SKU AX-330, during the morning shift. Reported by the operator. | Supplier response for this scenario. | inventory_supply | business | yuri_v3 |
| v3_e8ab89d71c30fb2a_c | choice | ["revenue_up","revenue_down","neutral","uncertain"] | [0.08,0.59,0.02,0.31] | A capacity expansion shortens the free tier from 14 to 7 days; additionally freezes hiring during peak season. Context: based on 3 years of history, with a new vendor onboarded. Reported by the operator. | Predicted outcome for this scenario. | policy_simulation | structured | yuri_v3 |
| workflow-controls-v1:security_incidents:42:172:v0:BLOCK EGRESS ASSET | noul | ["false","true"] | [1,0] | {"alert": {"activity": "endpoint_observation", "asset_id": "Case-557649b54115ebd7", "day": 372, "id": "Case-557649b54115ebd7:alert"}, "asset": {"environment": "staging", "id": "Case-557649b54115ebd7", "owner": "owner-Case-557649b54115ebd7", "redundancy": "replicated", "role": "ap… | Apply this local, explicitly stated workflow policy to the supplied facts. Each action is an independent yes/no judgment; respect the workflow action cardinality. Missing approval is not approval. Customer consent must match the pending proposal ID and latest customer message ID … | workflow-controls-v1/security_incidents | openjev | openjev_v2 |
| v3_a224ba06aed77d55_s | score | ["0","1","2","3","4","5"] | [0,0,0.01,0.69,0.3,0] | An invoice shows a currency mismatch; additionally references a canceled contract. Context: $23,400.00, during normal operations. Reported by the supervisor. | Rate fraud risk for this scenario on a 0-5 scale. | invoice_processing | business | yuri_v3 |
| v3_3a61fc66b29476cd_c | choice | ["database_query","api_call","file_read","web_search"] | [0.83,0.1,0.07,0] | An assistant pipeline must validate an email format; additionally requires exact-match lookup in a 5-row table. Context: on the first attempt, under reduced staffing. Reported by the on-call engineer. | Best tool for this scenario. | tool_selection | agent | yuri_v3 |
| v3_1f7a8ad2ba4c19e6_s | score | ["0","1","2","3","4","5"] | [0,0.01,0.08,0.54,0.36,0.01] | An nda governs disputes in a foreign jurisdiction; additionally includes a most-favored-nation clause. Context: for an EU data subject, at the end of the fiscal period. Reported by the review queue. | Rate contract risk for this scenario on a 0-5 scale. | legal_compliance | business | yuri_v3 |
| v3_bfeb287e82da70c3_s | score | ["0","1","2","3","4","5"] | [0,0,0.01,0.53,0.46,0] | Service catalog-sync correlates with the vendor status page incident; additionally shows clock skew of 800ms across replicas. Context: in production only, while legacy systems are still active. Reported by the on-call engineer. | Rate user-visible impact for this scenario on a 0-5 scale. | observability | technical | yuri_v3 |
| ours_0bf4a8bca7e3d1d9 | noul | ["false","true"] | [0.5,0.5] | American Wedding American Wedding (known as American Pie 3: The Wedding or American Pie: The Wedding, in some countries) is a 2003 American sex comedy film and a sequel to "American Pie" and "American Pie 2". It is the third (originally intended final) installment in the "Americ… | Is this memory relevant for answering the query? Query: What media organization which posts talks is Jesse Dylan a member of? | memory_relevance | knowledge | yuri_v1 |
| v3_86aaffc60f109364_n | noul | ["false","true"] | [0.9,0.1] | A hospital visit discloses abuse in the home; additionally discloses thoughts of self-harm. Context: in the first meeting, as part of a compliance sweep. Reported by the on-call engineer. | Is this scenario one where: this is within lay pastoral scope? | pastoral_care | theology | yuri_v3 |
| ours_271c52a93001af86 | noul | ["false","true"] | [0.5,0.5] | في عام 1901, تم ادخال سجلات القرص التي تبلغ 10 بوصة, تليها في عام 1903 بواسطة سجلات 12 بوصة. هذه يمكن ان تلعب لاكثر من ثلاث واربع دقائق على التوالي, في حين ان الاسطوانات المعاصرة لا يمكن ان تلعب الا لمدة دقيقتين. في محاولة للاعلان عن ميزة القرص, قدم اديسون اسطوانة amberol في عام … | Is this memory relevant for answering the query? Query: What was the name of the discs Edison introduced in 1909? | memory_relevance | knowledge | yuri_v1 |
| v3_b2e0aeb06cee8dfc_n | noul | ["false","true"] | [0.98,0.02] | A biosafety cabinet skipped the decontamination step; additionally used expired reagents from an unopened kit. Context: under the institutional biosafety protocol, under an active deadline. Reported by the monitoring system. | Is this scenario one where: the procedure was compliant? | lab_biosafety | biology | yuri_v3 |
| v3_2eb05b566d9b68b3_c | choice | ["highway_bypass","urban_direct","toll_road","wait_for_clearance"] | [0.66,0.06,0.18,0.1] | The delivery route requires an unpaved segment for 8km; additionally has a bridge with a 4.2m height limit. Context: under a 90-minute deadline, during the morning shift. Reported by the automated check. | Route choice for this scenario. | navigation_routing | spatial | yuri_v3 |
| v3_6d2f599234be84a5_s | score | ["0","1","2","3","4","5"] | [0,0,0.01,0.55,0.44,0] | A fire-suppression system shows refrigerant loss of 11%; additionally reports intermittent sensor faults. Context: the west campus, during normal operations. Reported by the operator. | Rate safety hazard severity for this scenario on a 0-5 scale. | facilities_maintenance | business | yuri_v3 |
| v3_cb05621849505be3_s | score | ["0","1","2","3","4","5"] | [0,0.01,0.03,0.5,0.46,0] | The approval chain has a built-in 3-day buffer; additionally requires signoffs from 4 people with 2 on leave. Context: with expedite fees available, as part of a compliance sweep. Reported by the on-call engineer. | Rate deadline pressure for this scenario on a 0-5 scale. | temporal_reasoning | spatial | yuri_v3 |
| v3_ec8b636bbeddce01_c | choice | ["producer_change","schema_drift","infrastructure","expected_variation"] | [0.17,0.77,0.06,0] | The nightly feed shows referential breaks against the dim table; additionally arrived 3 hours late. Context: feeding the quarterly report, shortly after a software update. Reported by the monitoring system. | Root cause for this scenario. | data_quality | technical | yuri_v3 |
| v3_d4a968676a6227b0_c | choice | ["treat","retest","image","refer_specialist"] | [0,0.05,0.55,0.4] | A symptom cluster has nonspecific findings with an unclear history; additionally reveals a 6mm non-calcified nodule. Context: in a 58-year-old smoker, with a new vendor onboarded. Reported by the operator. | Next step for this scenario. | diagnostic_reasoning | medical | yuri_v3 |
| v3_ffd73c9cb4a4a90b_n | noul | ["false","true"] | [0.64,0.36] | A systematic position denies the perpetual virginity explicitly; additionally uses analogy where the tradition uses identity language. Context: under the Augsburg confession, while legacy systems are still active. Reported by the supervisor. | Is this scenario one where: the formulation is precise? | doctrinal_reasoning | theology | yuri_v3 |
| v3_2edff76d6c516c87_s | score | ["0","1","2","3","4","5"] | [0.05,0.06,0.13,0.57,0.17,0.02] | The research abstract has the dissenting opinion in a footnote; additionally is already a summary of a summary. Context: for the executive audience, shortly after a software update. Reported by the monitoring system. | Rate compression difficulty for this scenario on a 0-5 scale. | summarization_control | knowledge | yuri_v3 |
| v3_93a93a1a416c056c_c | choice | ["raw","technical","summary","minimal"] | [0,0.11,0.88,0.01] | An executive summary uses plain language for a technical audience; additionally names an individual in a security failure. Context: for the board packet, during the quarterly review. Reported by the operator. | Detail level for this scenario. | communication_routing | agent | yuri_v3 |
| v3_8667e81e4e896073_n | noul | ["false","true"] | [0.72,0.28] | A fire-suppression system reports intermittent sensor faults; additionally shows refrigerant loss of 11%. Context: the west campus, during a scheduled maintenance window. Reported by the monitoring system. | Is this scenario one where: this requires an immediate shutdown? | facilities_maintenance | business | yuri_v3 |
| v3_f5ea8c6436e829e3_n | noul | ["false","true"] | [0.6,0.4] | An autonomous vehicle requires an unpaved segment for 8km; additionally shows a 40-minute delay from construction. Context: with a 3,500kg vehicle, during the system migration. Reported by the supervisor. | Is this scenario one where: a closure blocks the path? | navigation_routing | spatial | yuri_v3 |
| v3_284194c15726cc9c_c | choice | ["monitor","restore","intensive_remediation","escalate_regulator"] | [0.31,0.32,0.34,0.03] | A stream survey records invasive coverage at 65%; additionally shows canopy closure within historical range. Context: during the drought year, while legacy systems are still active. Reported by the automated check. | Intervention for this scenario. | ecology_environment | biology | yuri_v3 |
| v3_1fdf89207fb80259_s | score | ["0","1","2","3","4","5"] | [0,0,0.09,0.14,0.6,0.17] | A walk-in patient has a minor cut requiring attention; additionally has shortness of breath at rest. Context: age 24, no history, while legacy systems are still active. Reported by the review queue. | Rate acuity level for this scenario on a 0-5 scale. | clinical_triage | medical | yuri_v3 |
| snake-v1:52cfcb4a93898f7ecdc2e52e:action | choice | ["up","right","left"] | [0,1,0] | {"coordinates": "x increases right; y increases down; walls are outside the board", "direction": "up", "food": [4, 4], "game": "snake", "height": 6, "snake_head_first": [[1, 2], [1, 3], [1, 4], [1, 5], [2, 5], [3, 5], [3, 4]], "width": 6} | Choose a direction to keep the snake alive and collect food. The tail vacates on a move unless food is eaten; reversing direction is forbidden. | snake-v1 | openjev | openjev_v2 |
| ours_8a1c809d60d25e73 | noul | ["false","true"] | [0.5,0.5] | In his sophomore year, Kerry became the Chairman of the Liberal Party of the Yale Political Union, and a year later he served as President of the Union. Amongst his influential teachers in this period was Professor H. Bradford Westerfield, who was himself a former President of th… | Is this memory relevant for answering the query? Query: What was Kerry's role in the Yale Political Union as a junior? | memory_relevance | knowledge | yuri_v1 |
| workflow-controls-v1:security_incidents:42:154:v2:REVOKE ACCESS KEY | noul | ["false","true"] | [1,0] | {"alert": {"activity": "endpoint_observation", "asset_id": "Case-10a58539b3e7c214", "day": 134, "id": "Case-10a58539b3e7c214:alert"}, "asset": {"environment": "production", "id": "Case-10a58539b3e7c214", "owner": "owner-Case-10a58539b3e7c214", "redundancy": "single", "role": "dom… | Apply this local, explicitly stated workflow policy to the supplied facts. Each action is an independent yes/no judgment; respect the workflow action cardinality. Missing approval is not approval. Customer consent must match the pending proposal ID and latest customer message ID … | workflow-controls-v1/security_incidents | openjev | openjev_v2 |
| v3_1ee86a5f25b64921_c | choice | ["functional","segregation","population","computational"] | [0.01,0.49,0.11,0.39] | A brca1 variant is a missense change in a conserved domain; additionally is a silent change with no splice impact. Context: in an unaffected 70-year-old relative, during a scheduled maintenance window. Reported by the review queue. | Evidence category for this scenario. | variant_interpretation | genomics | yuri_v3 |
| v3_b8af3f3cfb23e32f_s | score | ["0","1","2","3","4","5"] | [0.01,0.01,0.06,0.55,0.36,0.01] | A meeting transcript has handwritten margin notes; additionally contains employee PII in attachments. Context: pending deletion review, shortly after a software update. Reported by the monitoring system. | Rate classification confidence for this scenario on a 0-5 scale. | document_classification | knowledge | yuri_v3 |
| v3_2bafeeecfa0711c0_c | choice | ["highway_bypass","urban_direct","toll_road","wait_for_clearance"] | [0.11,0.12,0.01,0.76] | The warehouse robot is the shortest path by 6 minutes; additionally crosses a flood-warning zone. Context: under a 90-minute deadline, shortly after a software update. Reported by the automated check. | Route choice for this scenario. | navigation_routing | spatial | yuri_v3 |
| v3_7750a26bbf7831d2_n | noul | ["false","true"] | [0.04,0.96] | A payment request matches no purchase order; additionally exceeds the PO by 12%. Context: €9,870.00, mid-incident. Reported by the operator. | Is this scenario one where: this requires manual review? | invoice_processing | business | yuri_v3 |
| v3_3f0ae9f2871c319e_n | noul | ["false","true"] | [0.95,0.05] | A dosage adjustment resumes anticoagulation pre-op without a plan; additionally continues a medication during documented pregnancy. Context: in a patient on 9 medications, during an external partner integration. Reported by the supervisor. | Is this scenario one where: this order is safe as written? | medication_safety | medical | yuri_v3 |
| v3_dccc1be1e015ff02_s | score | ["0","1","2","3","4","5"] | [0.06,0.09,0.09,0.28,0.39,0.09] | A hypothesis survived a preregistered adversarial test; additionally requires a new particle with no decay signature. Context: for a grant review, during the system migration. Reported by the monitoring system. | Rate novelty for this scenario on a 0-5 scale. | hypothesis_eval | science | yuri_v3 |
| v3_c9d76f6f615c2111_c | choice | ["ready","needs_context","not_ready","abort"] | [0.09,0.17,0.61,0.13] | The orchestrator exceeded its token budget by 3x; additionally completed with all invariants intact. Context: in a 4-agent fan-out, immediately after go-live. Reported by the supervisor. | Handoff verdict for this scenario. | multi_agent_coord | agent | yuri_v3 |
| ours_c5aeec81fa6ecf7c | noul | ["false","true"] | [0.5,0.5] | In the first quarter, New York took flight as QB Brett Favre completed an 18-yard TD pass to RB Leon Washington. In the second quarter, the Chiefs tied the game as QB Tyler Thigpen completed a 19-yard TD pass to TE Tony Gonzalez. The Jets would answer with Washington getting a 6… | Is this memory relevant for answering the query? Query: Who completed an 18-yard TD pass to RB Leon Washington? | memory_relevance | knowledge | yuri_v1 |
| ours_0768b4ebbcca9a2e | noul | ["false","true"] | [0.5,0.5] | a cheap flight ticket from sori to mestre starts from 216€. prices can change according to the available fares and the travel dates. if you book in advance your tickets from sori to mestre you can make the most of the available deals and save on your trip. | Is this memory relevant for answering the query? Query: how much is a flight ticket from sori to mestre? | memory_relevance | knowledge | yuri_v1 |
| ours_d0fc74b3617d413b | noul | ["false","true"] | [0.5,0.5] | In a recession, a decrease in consumer spending causes many businesses to lay off workers or even to close. Workers who lose their jobs in a recession usually cannot find new jobs. The result is an increase in the number of people who are jobless. Recovery from a recession is def… | Is this memory relevant for answering the query? Query: The conclusion about whether there would be a resulting net increase in spending would not follow if the taxpayers of the province would spend outside the province at least $300 million of any $600 million refunded to them t… | memory_relevance | knowledge | yuri_v1 |
| painting-geometry-v1:5330c25dbb33b411f407e704:palette:x0_y3 | choice | ["black: RGB [0, 0, 0]","white: RGB [255, 255, 255]","gray: RGB [128, 128, 128]","red: RGB [255, 0, 0]","lime: RGB [0, 255, 0]","blue: RGB [0, 0, 255]","yellow: RGB [255, 255, 0]","cyan: RGB [0, 255, 255]","magenta: RGB [255, 0, 255]","orange: RGB [255, 165, 0]","purple: RGB [128… | [1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0] | {"image_description": "Background is black. One solid red foreground rectangle includes exactly 3 <= x <= 5 and 0 <= y <= 3. All remaining pixels are background.", "width": 8, "height": 8, "coordinates": "Integer pixel coordinates. Origin (0,0) is top left; x increases right, y i… | Which palette color best represents pixel x=0, y=3? | painting-geometry-v1 | openjev | openjev_v2 |
| v3_5e971a65f032079c_s | score | ["0","1","2","3","4","5"] | [0,0,0,0.01,0.64,0.35] | A hospital visit shares a job loss and asks for prayer; additionally discloses abuse in the home. Context: in the first meeting, during the system migration. Reported by the operator. | Rate sensitivity required for this scenario on a 0-5 scale. | pastoral_care | theology | yuri_v3 |
| ours_5440535bd290119f | noul | ["false","true"] | [0.5,0.5] | The heroine, Sybylla Melvyn, is an imaginative, headstrong girl growing up in rural Australia in the 1890s. Drought and a series of poor business decisions reduce her family to subsistence level, her father begins to drink excessively, and Sybylla struggles to deal with the monot… | Is this memory relevant for answering the query? Query: Who is Sybylla sent to live with after her family's struggles? | memory_relevance | knowledge | yuri_v1 |
| v3_bd5095775ce6f0a5_s | score | ["0","1","2","3","4","5"] | [0,0,0.01,0.15,0.76,0.08] | An alignment result has a 4bp insertion in the coding sequence; additionally contains a stop codon mid-open-reading-frame. Context: at 30x mean depth, mid-incident. Reported by the monitoring system. | Rate call confidence for this scenario on a 0-5 scale. | sequence_analysis | genomics | yuri_v3 |
| workflow-controls-v1:agent_trace_observability:42:15:v0:NOT A BUG | noul | ["false","true"] | [1,0] | {"conversation": [{"role": "customer", "text": "Please update the selected resource once my approval is recorded."}], "feedback": "dissatisfied", "final_answer": "The operation failed; the resource was not changed.", "instructions": "Complete the customer's requested operation on… | Apply this local, explicitly stated workflow policy to the supplied facts. Each action is an independent yes/no judgment; respect the workflow action cardinality. Missing approval is not approval. Customer consent must match the pending proposal ID and latest customer message ID … | workflow-controls-v1/agent_trace_observability | openjev | openjev_v2 |
| v3_673c4626c7cf71ad_n | noul | ["false","true"] | [0.29,0.71] | A status report summarizes a resolved SEV2; additionally uses plain language for a technical audience. Context: in the public status page, immediately after go-live. Reported by the supervisor. | Is this scenario one where: the tone is appropriate? | communication_routing | agent | yuri_v3 |
| painting-geometry-v1:269e5c89a676d2f62c5ef4f0:rgb:x6_y7_green | noul | ["false","true"] | [1,0] | {"image_description": "Background is black. One solid cyan foreground rectangle includes exactly 6 <= x <= 7 and 2 <= y <= 4. All remaining pixels are background.", "width": 8, "height": 8, "coordinates": "Integer pixel coordinates. Origin (0,0) is top left; x increases right, y … | For pixel x=6, y=7, should its green channel be 255 rather than 0? | painting-geometry-v1 | openjev | openjev_v2 |
| v3_9886d0b149aeba9a_n | noul | ["false","true"] | [0.44,0.56] | An ldlr variant segregates with disease in the family; additionally is absent from population databases at 0.0004 frequency. Context: in a screening-only context, immediately after go-live. Reported by the on-call engineer. | Is this scenario one where: this variant is clinically significant? | variant_interpretation | genomics | yuri_v3 |
| v3_ed3cfc4ce318725b_c | choice | ["retain_permanent","retain_7y","retain_1y","destroy_now"] | [0.4,0.48,0.11,0.01] | An email thread cites the 2019 data-retention schedule; additionally contains a medical record reference. Context: pending deletion review, at the end of the fiscal period. Reported by the operator. | Handling for this scenario. | document_classification | knowledge | yuri_v3 |
| v3_cda41f8f6fa96b2a_s | score | ["0","1","2","3","4","5"] | [0.02,0.03,0.1,0.49,0.35,0.01] | An observed correlation has a circular dependency in its assumptions; additionally predicts a 15% effect with a 3% measured noise floor. Context: at the thesis defense, during an external partner integration. Reported by the automated check. | Rate evidence strength for this scenario on a 0-5 scale. | hypothesis_eval | science | yuri_v3 |
| v3_c9cb4dcba0f45eb8_c | choice | ["christology","soteriology","ecclesiology","theology_proper"] | [0.49,0.5,0.01,0] | A confessional statement holds two natures in one person; additionally teaches salvation apart from the stated means. Context: in the ecumenical dialogues, under an active deadline. Reported by the operator. | Doctrine area for this scenario. | doctrinal_reasoning | theology | yuri_v3 |
| v3_3fa156a98e376af4_n | noul | ["false","true"] | [0.86,0.14] | A policy exception request auto-renews without a termination right; additionally lacks a data-breach notification window. Context: for an EU data subject, during the morning shift. Reported by the review queue. | Is this scenario one where: the exception is grantable under policy? | legal_compliance | business | yuri_v3 |
| v3_d9cc5bfc07340487_s | score | ["0","1","2","3","4","5"] | [0.13,0.5,0.19,0.17,0.01,0] | The report generation depends on a task that runs at 03:00; additionally is idempotent and safe to retry. Context: every 4 hours, as part of a compliance sweep. Reported by the monitoring system. | Rate deadline feasibility for this scenario on a 0-5 scale. | scheduling | agent | yuri_v3 |
