FolDeX Evidence Audit: What 2,000 Hours of Real-Robot Data Actually Prove

The short answer

FolDeX is valuable because it makes long-horizon garment manipulation a physical evaluation problem, not only a simulation or demonstration problem. Its strongest evidence is a fixed real-robot protocol with randomized garment configurations, complete-task success rules, final-state quality, execution time, and a matched experiment showing better results after recovery trajectories were added.

The paper, posted on September 9, 2026, reports:

• more than 2,000 hours of real-robot experience;
• more than 20 manipulation tasks;
• more than 10 robot embodiments;
• synchronized views from two wrist cameras and one external camera, robot state, timestamped actions, language instructions, and episode metadata;
• four benchmark axes: recovery, task, scene, and embodiment;
• physical garment trials using held-out objects and randomized configurations;
• a FoldScore that combines final quality, complete-task success, and time;
• a FoldChallenge page intended to support external policy submission and standardized physical execution.

The paper also leaves important gaps. It does not publish a complete downloadable-corpus manifest or license for all 2,000 hours. The data-composition figure reports trajectory counts visually, but not a complete table of hours, trajectories, failures, recovery segments, tasks, operators, and robots. The authors’ recovery experiment does not disclose the amount of recovery data or intervention burden. Cross-task and cross-embodiment transfer remain preliminary, with naive mixing often failing.

The defensible conclusion is narrower than the headline: FolDeX establishes a useful benchmark design and promising author-run physical reference results. It does not yet establish that the full corpus is reproducible, that cross-robot transfer works, or that the reported policies are ready for unattended household operation.

What is actually being benchmarked

FolDeX centers on complete garment-handling sequences rather than one isolated grasp or fold. A representative episode begins with a garment in a basket. The robot retrieves it, places it on a table, flips and flattens it, performs multiple folds, then places or stacks the finished item.

That sequence matters because each stage changes the starting state of the next one. A poor retrieval can create an awkward table placement. A missed corner during flattening can make a later fold impossible. A policy can finish an early primitive while leaving a state from which the complete task cannot recover.

This is the same measurement problem covered in our diagnostic robot benchmark guide: final success should be paired with stage reach, state quality, failure location, and recovery. FolDeX adds a useful deformable-object case where shape, self-occlusion, entanglement, contact, and edge alignment continuously change.

The suite is folding-centered, not exclusively clothing and not evenly distributed across every task family. The paper lists garment categories such as shirts, skirts, pants, towels, and blouses. It also names other deformable and rigid tasks, including opening a bag, tying and untying a knot, sorting and picking food, inserting a plug, stamping, block construction, and cube picking. Rigid tasks are included mainly to test whether their data helps later deformable manipulation.

The 2,000-hour claim needs a manifest

The reported scale is substantial because the source describes physical robot experience, not rendered video duration or human handheld demonstrations. The named platforms include Aloha, YAM, Piper, Astribot, Franka, UR5, a custom mobile dual-arm system, and AgiBot G2, plus additional systems.

An hour is still not a common unit of information. The paper’s composition chart shows large differences in trajectories by embodiment and task. It does not provide a complete machine-readable accounting that lets a reader reconcile the headline with:

1. hours and trajectories for every task and embodiment;
2. successful demonstrations, failed rollouts, interventions, and recovery segments;
3. mean, median, and distribution of episode duration;
4. operator count, operator experience, and collection interface;
5. scene, object, and garment-instance coverage;
6. rejected, truncated, duplicated, or quality-filtered records;
7. training, validation, and evaluation partitions;
8. corpus versions, licenses, checksums, and access restrictions.

Those fields do not negate the reported scale. They determine what the scale means and whether another team can reuse it. Our guide to how robot training data is made explains why hours should be read beside provenance, synchronization, interventions, rights, and downstream effects.

What the common schema adds

The paper says each record can include synchronized visual observations, proprioception, timestamped actions, natural-language instructions, and episode metadata. Metadata can identify task, object category, object instance, scene, embodiment, operator, data source, completion status, and failure or termination reason.

That is a stronger design than a directory of videos. It makes several audits possible:

• check whether evaluation garments leaked into training;
• separate one operator’s habits from general policy performance;
• locate failed stages rather than discarding the whole episode;
• compare scenes without silently changing robot hardware;
• analyze whether recovery segments help only the states that generated them;
• preserve completion and termination reasons instead of treating every file as a success.

Version 1 describes these fields, but does not expose a complete public schema artifact or dataset card with field-level coverage. A schema can support audit only when users can inspect how consistently the corpus fills it.

Recovery data produced the clearest matched result

FolDeX’s recovery track starts with a policy rollout. A human intervenes when the policy enters a bad state, controls a correction, and records the recovery trajectory. The experiment combines those trajectories with independently collected demonstrations and trains a separately initialized policy.

The reference comparison uses one pi-zero policy with training-time real-time chunking, called RTC, across shirts, skirts, pants, and towels. The demonstration-only RTC multi-task baseline reported:

• 80.75% average complete-task success;
• 75.59 FoldScore.

After recovery trajectories from the same four garment categories were added, the recovery-augmented policy reported:

• 95.00% average complete-task success;
• 82.53 FoldScore;
• 90% success on shirts;
• 90% on skirts;
• 100% on pants;
• 100% on towels;
• an average successful completion time of 1 minute 58 seconds.

The average success gain is 14.25 percentage points under the reported setup. This is useful evidence because the before and after policies use the same broad model family, task set, and physical evaluation framework.

It is not a complete data-efficiency result. The paper does not state in the main record:

• how many recovery trajectories or hours were added;
• how often operators intervened to collect them;
• the number and duration of intervention segments;
• which failure states produced the largest gains;
• whether total training examples were held constant;
• whether an equal amount of new clean demonstration data was tested;
• how results vary across repeated policy training seeds;
• the complete cost of monitoring, resets, labeling, training, and physical evaluation.

The gain supports recovery-augmented learning in this experiment. It does not prove that recovery data is always more valuable than demonstrations or that the human burden is low. The robot intervention evidence guide shows the additional handoff, operator-time, false-trigger, recovery, and unassisted-reevaluation fields needed to make that claim.

Why the baseline details matter

FolDeX compares multi-task and task-specific policies under standard and RTC training. RTC conditions the model on action prefixes already committed for execution, with the goal of accounting for inference delay during training.

Across four garment categories, the paper reports:

Reference policy	Average success	FoldScore	Important qualification
RTC multi-task pi-zero	80.75%	75.59	Best FoldScore among the three main references
Standard multi-task pi-zero	72.50%	72.30	Lower success and quality-time composite
RTC task-specific pi-zero	82.50%	71.69	Highest average success, lower FoldScore

The ranking changes with the metric. The task-specific policies had the highest average success. The RTC multi-task policy had the highest FoldScore. On towels, RTC multi-task success was 76.67%, compared with 50.00% for the task-specific policy. On pants, the task-specific result was 80.00%, compared with 63.33% for RTC multi-task.

This is exactly why one average success rate is insufficient. A policy can complete more trials while producing less neat results or taking longer. A shared policy can help one garment and hurt another. The evaluation should preserve per-category results and the components behind the composite.

What counts as a successful physical trial

Unless otherwise specified, each garment category was evaluated in 30 independent physical trials with randomized garment configurations. A trial counted as successful only if the policy completed the full long-horizon sequence without human intervention or unrecoverable failure.

That definition is strong in three ways:

1. it excludes assisted completions from autonomous success;
2. it evaluates the dependent sequence rather than a selected successful stage;
3. it provides a physical trial denominator by garment category.

Final-state quality was scored on a six-level scale from 0 to 5. The rubric considers shape, compactness, flatness, stability, and edge alignment. At least two evaluators independently assessed qualitative scores, and a score was recorded only when they agreed. The same evaluators were used across methods.

The remaining audit questions are who resolved disagreement, whether evaluators were blinded to method, inter-rater statistics, garment-level results, excluded attempts, reset rules, and videos for every scored rollout. Agreement is better than one undisclosed judge, but it is not the same as a fully reproducible measurement instrument.

FoldScore is more informative than success, but it is still a choice

FoldScore allocates:

• 35 points to average normalized final-state quality;
• 35 points to complete-task success;
• 30 points to speed relative to the maximum episode duration.

The time component is zero when no trial succeeds. This prevents a fast failure from receiving efficiency credit.

The metric addresses a real weakness in binary success. Two policies can both finish while producing different folds, or one can be much slower. Still, a composite score embeds value judgments. A user who values throughput more than neatness might choose different weights. A laundry application may care about stack stability, wrinkles, garment damage, energy, intervention burden, and downstream machine compatibility, none of which is fully represented by the same three terms.

Reports should therefore publish FoldScore with its components, not replace them with the composite. FolDeX does this in its reference tables, which makes the ranking interpretable.

Held-out physical objects and controlled initialization

The paper says FoldChallenge evaluates externally submitted policies with standardized task definitions, held-out physical objects, controlled initializations, and a unified execution protocol. This can reduce three common sources of benchmark inflation:

• tuning directly on the exact evaluation garment;
• selecting easy initial configurations;
• changing execution settings between models.

The paper also says evaluations use the same physical tasks, garment instances, initialization procedure, maximum episode duration, and robot execution protocol.

The current evidence is partly a protocol description. The paper does not publish a complete versioned list of held-out object rules, initialization distributions, hardware and software images, submission limits, compute policy, evaluator blinding, conflict process, or all leaderboard rollouts. It says the platform reports auditable rollout videos, but version 1 also notes that some identifying access information was omitted during review even while another page points to the Midea platform.

A hosted benchmark becomes durable when the public record includes a stable protocol version, policy-submission contract, hardware calibration procedure, all-run logs, change history, and archival result files.

Cross-task transfer remains unresolved

FolDeX asks whether rigid-object data can help deformable manipulation and whether models can adapt with limited target-task data. The preliminary result is cautionary.

The authors say direct joint training often introduced substantial interference in action generation. Continued training frequently caused previously learned actions to be forgotten. Policies could sometimes complete short stages, but rarely preserved reliable performance across the full long-horizon task.

This does not mean cross-task transfer is impossible. It means the paper does not provide a successful general recipe. Task labels, observation similarity, and shared grasp-place patterns were not enough to make naive data reuse reliable across the full sequence.

The appropriate future evidence is a controlled scaling curve. Hold the target-task evaluation fixed, vary source-task data and target-task data independently, and report where transfer helps, does nothing, or causes negative transfer.

Cross-scene transfer looked easier than cross-embodiment transfer

The scene track changes lighting, visual background, spatial layout, and surrounding environment. The authors report stronger robustness to these changes than to task-dynamics or embodiment changes.

This is a qualitative preliminary observation, not a complete numerical generalization table. A reader cannot yet compare performance by lighting shift, camera pose, workspace geometry, clutter, or background.

Cross-embodiment tests were harder. The robots differ in kinematics, workspace, camera placement, controls, and action representations. Under naive joint and continued training, the authors report severe action-space interference and catastrophic forgetting, with policies generally unable to complete the full task reliably.

That negative result is useful. A dataset containing ten robots does not itself prove transfer to ten robots. The embodiment gap guide explains why source diversity, target exposure, action mapping, calibration, control, and robot-specific evaluation must remain separate.

RoboDrop provides the quality counterpoint

FolDeX asks how recovery and heterogeneous physical experience can be reused. RoboDrop, a separate September 2026 paper, asks which demonstrations should be removed before VLA post-training.

RoboDrop uses a one-epoch warm-up run to score candidate samples by local gradient compatibility with clean validation examples matched by task semantics and visual context. It aggregates sample scores at the episode level. An optional procedure fits one-component and two-component Gaussian mixtures, uses BIC to select between them, then removes sufficiently confident low-score episodes without assuming the contamination rate in advance.

On four real-robot tasks, Hang Cup, Tidy Pen, Stack Cup, and Drawer, fresh pi-zero-point-five Base policies were evaluated with 20 rollouts per task. The reported average success rates were:

Curation method	Average real-robot success
Unfiltered data	35.00%
Behavior Retrieval, fixed 20% removal	58.75%
RoboDrop, fixed 20% removal	63.75%
RoboDrop, automatic removal	67.50%

This is evidence that data quality can dominate raw volume in a bounded post-training setup. It is not a direct FolDeX comparison. The robots, tasks, data, model training, and evaluation protocols differ. RoboDrop also relies on a clean reference set, which the authors identify as a remaining expert-effort requirement.

Together the papers support a practical rule: count physical hours, but also identify recovery value, bad supervision, retained-data fraction, held-out evaluation, and policy gain. A larger corpus can contain more useful edge cases and more harmful mistakes at the same time.

An evidence card for large real-robot datasets

Before treating a robot-data release as a foundation for model progress, reconstruct this record.

Field	Minimum useful disclosure
Scale	Raw and accepted hours, trajectories, frames, tasks, scenes, objects, operators, and embodiments
Provenance	Demonstration, autonomous rollout, assisted rollout, intervention, recovery, reset, and failure source
Synchronization	Camera, state, action, force, control rate, latency, clocks, and dropped-data handling
Composition	Per-task and per-robot distributions, duration, success, failure, intervention, and termination reasons
Rights	Dataset license, source licenses, consent, site restrictions, and commercial-use terms
Splits	Held-out tasks, object instances, scenes, operators, embodiments, and contamination checks
Training	Model and checkpoint, data mixture, sample weighting, compute, seeds, and action interface
Evaluation	Hardware, task rubric, initialization, timeout, attempts, exclusions, resets, and all-run records
Human burden	Collection, monitoring, intervention, labeling, reset, scoring, and maintenance time
Effect	Fixed-policy ablations, uncertainty, negative transfer, failure classes, and cost per verified gain

FolDeX fills meaningful parts of the task, observation, action, metadata, physical-evaluation, and metric rows. The next release should make the scale, access, rights, composition, recovery burden, and versioning rows equally inspectable.

Evidence boundaries

Claim	Classification	Confidence	Boundary
FolDeX contains more than 2,000 hours, 20 tasks, and 10 robot embodiments	Author-reported corpus scale	High that the paper makes the claim	Complete manifest, distribution, license, and independent verification are not public in version 1
FolDeX is entirely real-robot data	Author description	Medium-high	Named platforms and collection schema support the description, but the complete corpus cannot yet be audited externally
Recovery data raised average success from 80.75% to 95.00%	Matched author-run physical experiment	Medium-high	Recovery volume, operator burden, equal-size data baseline, and repeated training seeds are not disclosed
The benchmark proves cross-task transfer	Not established	High	Naive joint and continued training often caused interference or forgetting
The benchmark proves cross-embodiment transfer	Not established	High	Preliminary policies generally failed complete tasks under naive multi-robot reuse
Cross-scene transfer is solved	Not established	High	The reported observation is qualitative and lacks a complete per-shift table
FoldChallenge makes comparisons fair	Protocol goal with partial public evidence	Medium	Durable fairness depends on versioned rules, calibration, submission controls, all-run logs, and governance
RoboDrop proves less data is always better	Not supported	High	Its gain is specific to four tasks, one real-robot data setting, a clean reference set, and author-run evaluation
FolDeX establishes household deployment readiness	Not supported	High	No customer operation, unattended hours, safety case, maintenance record, or production economics are reported

What to watch next

1. A downloadable, versioned FolDeX manifest with task, robot, hour, trajectory, operator, intervention, recovery, and failure distributions.
2. Dataset terms, license compatibility, consent and site rules, checksums, and stable hosting.
3. Recovery-data ablations by trajectory count, duration, failure type, and matched demonstration budget.
4. Multiple training seeds and uncertainty for success, quality, and completion time.
5. Garment-instance results, scorer agreement, excluded attempts, resets, and every rollout video.
6. Numeric cross-scene tables for lighting, background, layout, camera, and clutter changes.
7. Successful and failed cross-task scaling curves under fixed target evaluation.
8. Cross-embodiment methods that beat target-only baselines while reducing target-robot data.
9. External policy submissions with frozen benchmark versions and comparable execution records.
10. Tactile, force, damage, safety-stop, intervention, energy, and maintenance measurements for contact-rich folding.

Verdict

FolDeX advances the public evaluation record in three useful ways. It targets complete deformable-object sequences on physical robots. It separates success from final quality and speed. It treats interventions, recoveries, task variation, scene variation, and robot variation as data questions that need controlled tests.

Its best current result is the recovery-augmented physical comparison. The improvement from 80.75% to 95.00% average success is meaningful inside the authors’ four-garment protocol. The result becomes more actionable when the amount and cost of recovery data, training uncertainty, and complete rollout record are public.

The transfer results are equally important because they are not uniformly positive. Scene changes appeared more manageable, while naive task and embodiment mixing caused interference and forgetting. Ten robot bodies in one corpus do not automatically become one transferable policy.

FolDeX should be read as a promising physical benchmark and a research agenda for better data reuse. Its next proof point is not a larger hours headline. It is a versioned, accessible corpus and external policies that improve complete-task performance under the same held-out physical protocol.

Frequently asked questions

What is FolDeX?

FolDeX is an author-reported real-robot dataset and benchmark for long-horizon manipulation of deformable objects, centered on garment folding. Its four study axes cover recovery-data use, transfer across tasks, transfer across scenes, and transfer across robot embodiments.

Is the entire 2,000-hour FolDeX dataset publicly downloadable?

That is not established by the first paper version. The paper reports more than 2,000 hours of real-robot experience and points to a public FoldChallenge evaluation page, but it does not provide a complete versioned dataset manifest, download license, file count, or checksum record for the full corpus.

Did recovery data improve garment-folding performance?

In the authors' matched four-category experiment, adding recovery trajectories to an RTC multi-task pi-zero policy raised average complete-task success from 80.75% to 95.00% and FoldScore from 75.59 to 82.53. The paper does not isolate recovery-data volume, intervention frequency, operator time, or repeated training-seed uncertainty.

Does FolDeX prove that data transfers across robots?

No. The benchmark defines a cross-embodiment track, but the authors report severe action-space interference and catastrophic forgetting under preliminary naive joint and continued-training tests. They do not publish a successful general cross-robot transfer result in version 1.

What does FoldScore measure?

FoldScore gives 35 points to normalized final-state quality, 35 to complete-task success, and 30 to completion speed relative to a task timeout. It exposes quality and efficiency that binary success misses, but still depends on the scoring rubric, evaluators, task definition, and fixed weight choices.

Is FolDeX deployment evidence?

No. FolDeX is a research benchmark with author-run physical experiments and a hosted evaluation proposal. It does not establish a commercially available product, a customer pilot, unattended operation, or recurring production deployment.