2024/03/20 by Yuxuan Luo, Zekun Wu, Luo, Yuxuan +3
Computer Science · Engineering · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Hand Gesture Recognition Systems #Handwritten Text Recognition Techniques #Human Motion and Animation #Robotics (cs.RO)
paper · pdf · doi:10.48550/arxiv.2405.15776
openalex publication_date 2024/03/20 · openalex created_date 2024/05/29 · openalex updated_date 2026/07/28
Human-like planning skills and dexterous manipulation have long posed challenges in the fields of robotics and artificial intelligence (AI). The task of reinterpreting calligraphy presents a formidable challenge, as it involves the decomposition of strokes and dexterous utensil control. Previous efforts have primarily focused on supervised learning of a single instrument, limiting the performance of robots in the realm of cross-domain text replication. To address these challenges, we propose CalliRewrite: a coarse-to-fine approach for robot arms to discover and recover plausible writing orders from diverse calligraphy images without requiring labeled demonstrations. Our model achieves fine-grained control of various writing utensils. Specifically, an unsupervised image-to-sequence model decomposes a given calligraphy glyph to obtain a coarse stroke sequence. Using an RL algorithm, a simulated brush is fine-tuned to generate stylized trajectories for robotic arm control. Evaluation in simulation and physical robot scenarios reveals that our method successfully replicates unseen fonts and styles while achieving integrity in unknown characters.