Neuro-Symbolic Reasoning Improves LLM Accuracy by 12-18%
Generating natural language explanations then verifying them symbolically outperforms chain-of-thought prompting.
Underlying Paper
Reasoning with Natural Language Explanations: A Neuro-Symbolic Approach
We propose ExplainThenReason (ETR), a neuro-symbolic framework that decomposes complex reasoning tasks into explicit natural language explanation generation followed by symbolic verification. ETR first generates step-by-step explanations using an LLM, then translates these into a formal logic program for verification. On challenging reasoning benchmarks (ARC, FOLIO, ProofWriter), ETR improves accuracy by 12-18% over chain-of-thought prompting while providing provably correct reasoning chains.
The pursuit of scalable generative models has driven a wave of architectural innovation, yet the quadratic cost of attention in transformer-based diffusion models remains a fundamental bottleneck. This paper introduces a compelling alternative: replacing the attention backbone entirely with structured state space models (SSMs).
Core Contribution
The authors demonstrate that Mamba-style SSMs can serve as drop-in replacements for attention layers in the U-Net architecture commonly used for diffusion. The key insight is that the selective scan mechanism of modern SSMs naturally captures the multi-scale spatial dependencies required for high-quality image generation.
Technical Approach
The architecture, dubbed DiS (Diffusion with State Spaces), modifies the standard DiT (Diffusion Transformer) by replacing each attention block with a bidirectional SSM layer. The authors introduce a novel "cross-scan" strategy that processes image patches along four spatial directions simultaneously, aggregating the results to capture both local texture and global structure.
Results and Analysis
On ImageNet 256×256 unconditional generation, DiS achieves an FID of 2.67, comparable to DiT-XL/2 (FID 2.27) while requiring 3.2× fewer FLOPs per denoising step. The gap narrows further at 512×512 resolution, where DiS achieves FID 3.41 vs. DiT's 3.04 — a marginal quality difference that may be acceptable given the substantial computational savings.
Training convergence is notably faster: DiS reaches its best FID in approximately 400K steps compared to DiT's 700K steps under identical training budgets. The linear-time scaling also enables generation at resolutions the transformer variant cannot practically reach without additional engineering.
Evidence Box
strongKey Claims
- •Explicit explanation generation + symbolic verification improves reasoning
- •Provably correct reasoning chains via formal logic translation
Key Results
- •+12-18% accuracy over chain-of-thought on ARC/FOLIO/ProofWriter
- •Reasoning chains verified correct in 94% of cases
- •Generalizes across reasoning types (deductive, abductive, analogical)
Limitations & Caveats
- •Symbolic verification step adds 2-3× latency
- •Logic translation fails on ~6% of generated explanations
- •Requires domain-specific logic templates
