Loading SOILIE-3D

Research record

Research methods, evidence, and limitations.

This page documents the research question, implemented method, comparison evidence, interface extension, experimental controls, and known limitations.

Research question

Can a new scene be constructed from remembered relations rather than recalled as a whole?

SOILIE treats measurements from labelled RGB-D observations as a compact relational memory. It extracts object triplets, distances, angles, and scale estimates, then recombines compatible observations for a new collection of objects.

This is not a neural image generator, a theory of every form of imagery, or a validated simulation of a human mind. The intermediate representation is legible: the sampled relations, rule order, placements, room boundaries, and rendered result can all be inspected.

The SOILIE-3D article currently being prepared for publication is authored by Mike Cichonski, Isil Sanusoglu, Michael O. Vertolli, and Jim Davies. It presents the research motivation and implementation; this page distinguishes implemented behaviour from claims that still require evidence.

What SOILIE-3D implements

A sequence of explicit, cognitively motivated rules.

The article draws each rule from work on scene perception, imagery, memory, or intuitive physics. Encoding those ideas makes them testable; it does not by itself establish that people construct images in the same way.

01 · Sources

Two RGB-D scene corpora

SOILIE-3D draws relational measurements from Princeton SUN3D and Washington RGB-D Scenes v2. The Washington material contributes 14 indoor scenes covering nine labelled object classes, including tabletop objects, chairs, sofas, and tables.

02 · Translation

One common label space

SUN3D supplies 2D pixel labels while Washington supplies 3D point-cloud labels. Camera pose and intrinsic estimates project the Washington labels into 2D bounding boxes across 30 evenly distributed frames per scene.

03 · Relational memory

Triplets, distances, and angles

Polygon centroids become A–B–C object triplets with measured distances and angles. Compatible triplets can then be recombined instead of retrieving an intact source room.

04 · Geometry

Fuzzy scale and directed orientation

Objects use five overlapping size categories sampled toward a category centre. Assets are normalized upright, backs can face nearby walls, and front-any or front-front pairs orient related objects toward one another.

05 · Structure

Boundaries, grounding, and support

Floors and walls are generated as implicit architecture. Objects are grounded, while simplified support rules use surface shape and categorical size as proxies when resolving stacked arrangements.

06 · Depiction

Order changes the imagined view

Objects arrive one at a time. SOILIE-3D’s focus schedule keeps the newest object fully present while earlier objects lose opacity, detail, brightness, and saturation. The formation camera uses a third-person viewpoint and turns toward each new focus. Although the article proposes a five-item visibility limit, the current alpha schedule does not strictly hide older items once six or more have appeared.

Why ordinary and imagined renders both exist

The ordinary render keeps the geometry opaque for inspection. The imagined render exposes the model’s recency and vividness hypotheses without confusing a visual effect with a measured property of human imagery.

Model representation · 01

A scene moves through four representations.

Select a stage to see what information exists at that point in the experiment.

SOILIE processing pipelineAn interactive four-stage diagram showing source observations becoming relationships, a coordinate layout, and a rendered room. RGB-D observations Relation triplets Model arrangement 3D proposal

Stage 01 · Observe

Two label formats become comparable observations.

SUN3D supplies 2D labels. For Washington RGB-D Scenes v2, camera pose and intrinsic estimates project 3D point labels into 2D bounding boxes from 30 sampled frames per scene.

RGB-D frame + labels → object centroids

Evidence status

What the current work shows and what it does not.

The article presents qualitative inspection of generated scenes. Its examples show the rules operating together, but they do not establish correspondence with human imaginative processes.

Observed in outputs

Inspectable rule execution

Generated examples demonstrate varied arrangements, relationship-aware rotations, categorical scale, room boundaries, stacking attempts, colour sampling, and ordered visual fading.

Machine evidence

Completed spatial arrangements

The comparison protocol evaluates object overlap, boundary intrusion, support, connected space, generation speed, and cost. Shared eligible subsets and different native workloads require separate interpretation. New results are being prepared.

Current research position

A generator inspired by scene-perception priors

SOILIE-3D is best read as a transparent, cognitively inspired procedural scene-generation framework whose stronger cognitive claims remain open hypotheses.

Relational structure · 02

Compatibility is a connected graph, not a keyword match.

In this diagram, a link means that the pair participates in at least one recorded A–B–C triplet. A triplet is only recorded when one labelled frame contains three different objects, allowing two distances and their angle to be measured together. Two objects may therefore have appeared in the same frame without gaining a link if no eligible third object was present to complete the triplet.

Choose an object to highlight its triplet-supported neighbours. A requested set must remain connected through these recorded relationships.

Select a node to inspect its observed neighbours.

Interactive object relationship graphA graph connecting bed, desk, lamp, chair, table, and book when pairs participate in recorded three-object triplets. bed desk lamp chair table book

Room-generation comparisons

Geometry, timing, and evaluation methods.

The comparison protocol measures object overlap, room boundaries, support, usable space, generation time, and cost. These are specific properties of indoor arrangements, not a model's overall intelligence or similarity to human imagination.

Replacement results are being prepared

A new SOILIE room corpus is being prepared for generation and validation. Quantitative comparisons and AI review will be published together after their measurements and evidence checks are complete.

The 2024 demonstration archive remains available for inspecting example scenes.

Methods and limits

Geometry is measured on completed rooms before the website’s optional room-size adjustment. Comparisons require documented room selection, object counts, physical units, and execution conditions. AI judgments remain separate from human validation.

Geometry, units, and measurement limits

The mesh diagnostic uses evaluated triangles. Disjoint enclosing boxes prove separation; intersecting closed meshes permit Boolean volume measurement. Open or non-manifold meshes use surface-crossing tests instead. No surface crossing does not establish an enclosed volume or rule out containment inside an undefined interior. Object-envelope overlap remains a separate cross-source diagnostic for releases such as LayoutGPT that provide boxes but not meshes.

Support uses sampled mesh surfaces where geometry and physical units are available; contact alone does not establish stability. Clearance measures the largest connected floor region where the centre of a 0.6-metre-wide, 1.8-metre-tall cylinder can move. It is a narrow circulation diagnostic, not an accessibility certification. Measurements requiring meshes, physical scale, or a defined room boundary remain unavailable when an external source release does not provide them.

Generated architecture is excluded from furniture-to-furniture overlap. Boundary checks use each model's generated or supplied room before cutaway rendering and before the website's optional room-size adjustment. Auto-built boundaries and fixed input floor plans remain different tasks.

Selection and evaluation controls

Bedroom and living-room requests are assessed separately. Repeated labels remain separate object instances. Documented aliases normalize different names for the same class; window, blinds, and curtain components form an architectural assembly and are excluded from furniture measurements.

Cross-model comparisons use shared eligible object counts and documented matching rules. Missing observations remain unavailable rather than zero. AI review requires frozen scene evidence, prompts, reviewer configuration, and side assignments; those judgments do not establish human agreement.

Optional interface extension

Room-size control changes boundaries after placement.

Room-size control is a web-interface addition, not part of the core SOILIE-3D model. The engine first completes the model’s placement unchanged. It then measures that arrangement and redraws only the floor and walls. This illustration lets you explore anchor choices manually; the deployed wrapper infers an anchor from the model’s wall-oriented objects.

6.4 m
0.0 m
0.0 m

Drag the cyan boundary to place it freely. Arrow keys move it by 0.1 m; hold Shift for 0.5 m.

Exact fit: the requested boundary contains every measured footprint.

requestedactual roomobject envelope
Interactive anchored room fitting illustration The requested boundary can be moved or anchored to a bed corner or desk wall. A solid boundary shows the collision-free room the optional fitter would use. bed table desk Measured object envelope

Experiment guide

Procedure for the public research demonstrator.

The public engine is a bounded research demonstrator. Basic mode needs no configuration; Advanced mode maps the terminal generation choices onto the same model functions.

Open the engine
  1. 01

    Choose a question

    Generate a random set, provide a specific list, or sample an available SOILIE-3D bedroom, living-room, or kitchen combination. The model’s bathroom combination file is empty, so that preset is unavailable.

  2. 02

    Follow processing stages

    Queued, placing, rendering, and animating reflect worker progress. Missing-triplet and rendering failures are reported with their causes.

  3. 03

    Inspect and reproduce

    Compare ordinary and imagined views, then download the coordinates, normalized request, formation animation, and result manifest.

Inputs and controls

The interface exposes six experiment inputs.

Generation mode

Random samples a connected object set. Specific objects tests the labels you choose. Room type samples a supported room vocabulary.

Objects and scenes

Each scene contains three to six objects. A request may contain one to three scenes. Random mode can reuse one set or resample it for every scene.

Duplicates

Random mode may permit repeated classes. Specific mode accepts a repeated label directly; every copy receives a stable, independent instance ID.

Room size

The web demonstrator applies optional auto and custom sizing after model placement. Auto mode tightly wraps the final layout around a semantic anchor. Custom mode accepts width and depth from 1.0 to 20.0 metres; height follows the tallest object and a 2.4-metre minimum.

Seed

An integer seeds the random generators used by SOILIE-3D for object selection, triplet sampling, dimensions, colours, asset choice, and render order. The container also fixes Python’s hash seed. Leave it empty when you want the service to choose one.

Same objects

With multiple random scenes, reuse compares different placements of one collection. Turning it off tests independently sampled collections.

Reading a result

Each result includes renders and placement evidence.

Ordinary view

The opaque bird’s-eye reference render of the proposed geometry.

Imagined view

SOILIE-3D’s recency-weighted third-person sequence, in which placement order changes visual emphasis.

Placement evidence

The normalized request, model result rows, final room dimensions, seed, warnings, provenance, and links to every artifact.

Adjusted to fit

This notice appears when the completed model arrangement extends beyond a requested width or depth. The boundary keeps every feasible requested axis and expands only deficient axes. No interior object is moved, rotated, or rescaled by this web extension.

Questions and failures

Failure codes identify reproducible pipeline outcomes.

Use the stable code shown with an error when reporting a reproducible issue.

Why is my object combination unsupported?

The labels are known, but the source observations do not connect them into one relation graph. Try a suggested combination or replace a disconnected object.

Why did my room become larger?

The model arrangement is completed before custom sizing is applied. If its measured bounds do not fit, the web extension keeps any sufficient requested axis and expands only the deficient axes away from an inferred wall anchor. It does not revise SOILIE-3D’s interior placement.

Why is the demo temporarily unavailable?

The public experiment is rate-limited to six scenes per visitor each UTC day, 20 scenes across the service per day, and 600 per month. Rendering capacity is limited to two concurrent scenes.

How long are generated scenes available?

Generated artifacts expire after seven days unless their creator adds them to the public gallery. Download anything you want to keep privately.

What does SOILIE_GENERATION_FAILED mean?

The SOILIE-3D pipeline could not complete the requested scene, for example because an ordered triplet needed by its coordinate construction was unavailable. The service reports the cause with the failed job.

What accessibility settings are supported?

The interface supports keyboard use, visible focus, scalable layouts, accessible status updates, and reduced motion. With reduced motion enabled, decorative animation stops and homepage formation previews become still images.

Limits and open questions

Known limitations and unresolved questions.

01

Dataset dependence

Object compatibility reflects two low-resolution, incompletely labelled RGB-D corpora without complete 360-degree coverage. Missing or noisy observations can distort size, distance, and angle estimates.

02

Proxy assets and simplified physics

The Blender meshes represent object classes rather than the exact recorded objects. Bounds, categorical weight, and flat-surface checks do not capture every affordance or stability condition.

03

Third-person camera

SOILIE-3D’s formation sequence uses the article’s third-person viewpoint and turns toward each focal object. It does not model a moving observer, eye movements, embodied action, or a measured first-person view of mental imagery.

04

Order can expose temporary errors

An upper object may be placed before its support and appear to float until a later step. Some asset transparency and texture choices can also produce visual artifacts.

05

Cognitive interpretation

Fading, focal lighting, item visibility, categorical size, and placement sequence are computational hypotheses. The code’s current alpha schedule also does not strictly enforce the article’s proposed five-item visibility limit. No behavioural study has yet shown that these mechanisms predict subjective imagery.

06

Human evaluation

Spatial geometry, generation time, cost, and AI judgments assess specific computational outcomes. Human ratings and cognitive-process measures are still needed before making claims about human judgement or imagination.