概述

在过去的十年中,研究社区在3D几何与视觉方向付出了巨大的努力,并取得了很多令人印象深刻的突破性成果。然而,3D研究与大规模工业落地仍然存在较大的鸿沟。可能的主要原因有:第一,现有的大型3D数据集的CAD模型主要来自于网络,所以存在工业级质量模型稀少且无纹理或纹理信息简单的不足。第二,目前数据集的图片与CAD模型匹配都是伪人工匹配,且数量稀少。基于这些潜在缺陷,当前大型3D数据集可能不足以支持工业级的3D模型重建与纹理恢复等领域的深入细致研究。

本workshop的目的是总结目前SOTA的3D几何视觉研究,启发高质量3D模型理解和重建等领域的创新研究,并建立学术研究与工业落地的桥梁。为了实现这一目的,因此我们开源针对家居场景的3D-FUTURE数据集(3D FUrniture shape with TextURE)。该数据集包含20,240幅工业渲染器渲染的非常逼真的场景图,取自5000个不同的室内场景,这些场景都是根据专业设计师的房屋搭配布局而来;其中涉及9,992个精细的高质量3D家具模型,并提供高清且饱含信息的纹理以及各种属性标签。基于3D-FUTURE数据集,我们将举办阿里巴巴3D人工智能挑战赛,该挑战赛包含基于图像的3D形状检索,三维物体重建,与实例分割。我们鼓励参赛选手提交基于已发表论文的创新方法,或是正在审核论文的技术。

竞赛项目

比赛项目全部基于3D-FUTURE数据集, 数据积累沉淀于阿里巴巴官方家装家居设计平台-躺平设计家。开源数据由阿里巴巴淘系技术部组织建立。比赛平台由阿里巴巴天池搭建。

基于图像的3D形状检索

在该项挑战赛中,参赛者被要求根据2D图片在给定3D池中检索出对应的3D模型。我们期望通过这项比赛来促进具有鲁棒性的形状检索发展,即在图片中物体存在轻微遮挡以及具有复杂背景的情况下实现较高精度的检索。我们将用TopK召回率与TopK平均F-Score作为主要性能衡量指标。更多信息详见这里

基于单张图像的三维重建

在该项挑战赛中,参赛者有要求从单张RGB图像重建对应的3D模型。该项目非常具有挑战性,因为输入的RGB图像中的主物体可能存在轻微遮挡以及少部分残缺。Chamfer Distance以及F-score将作为重建结果质量的评价标准。更多信息详见这里.

实例分割

在该项挑战赛中,参赛者被要求对渲染的场景图进行实例分割。我们举行这项竞赛是因为它可以启发基于视觉的图像编辑与生成,也是家居场景理解的基础,因此有潜力革新工业生产。例如,可能部分取代昂贵的渲染过程。该项挑战赛的评估指标为Mask Average Precision (mAP)。更多信息详见 这里.

奖励

第一名: $1500

第二名: $1000

第三名: $500

IJCAI-PRICAI 2020 Workshop

报告演讲

IJCAI-PRICAI 2020 Workshop 报告

共同作者

竞赛榜单


重要日程节点

  • 2020-12-31        阿里巴巴 & IJCAI-PRICAI 2020 3D人工智能挑战赛Workshop将于 2021.1.8 (UTC) 在线上举办.

  • 2020-12-14        阿里巴巴 & IJCAI-PRICAI 2020 3D人工智能挑战赛榜单发布.

  • 2020-09-22        发布3D-FUTURE科技报告

  • 2020-03-30        竞赛开启报名,部分训练及测试样例发布.

  • 2020-04-03        完整训练集与验证集发布.

  • 2020-05-20        评估代码发布,工具脚本发布,相关baseline发布。

  • 2020-07-19        竞赛截止注册,测试集发布 (23:59:59 PST)。

  • 2020-07-24        竞赛提交截止时间 (23:59:59 PST)。

  • 2020-08-07        竞赛相关报告提交日期。

  • 2020-08-21        竞赛优胜者宣布。

Workshop 内容

How to enter the workshop zoom meeting:

  1. Login to the virtual chair platform. (https://www.virtualchair.net/events/ijcai-pricai-2020)
  2. Use the “Calendar” feature on the left side of your screen to locate the workshop room W35 . Click on the Room Name, and a line will appear to guide you.
  3. When entering the room you will see the message ‘press X to enter zoom call’. This will bring you directly into the zoom meeting


Morning Session (January 8, UTC Time)

12:00 am - 00:15 am Welcome and Introduction
00:15 am - 01:15 am Keynote Talk 1 (Tatsuya Harada).
Understanding 3D Structure from Limited Supervised Information and Tracking Non-Rigid 3D Objects
Video
01:15 am - 01:55 am Keynote Talk 2 (Jun-Yan Zhu)
3D-aware Image Synthesis and Editing
01:55 am - 02:45 am Keynote Talk 3 (Rongfei Jia)
Beginning 3D Studies through 3D-FUTURE and 3D-FRONT Benchmarks
02:45 am - 03:00 am Coffee Break.
03:00 am - 04:00 am Contributed talk (1) - 3D Object Reconstruction from A Single Image
  • Winner 1 - 3D Object Reconstruction from A Single Image with Point Cloud Deformation
  • Winner 2 - 3D Object Reconstruction from A Single Image based on point cloud Generation
  • Winner 3 - 3D Object Reconstruction from A Single Image Based on Instance Segmentation
04:00 am - 05:00 am Contributed talk (2) - Instance segmentation
  • Winner 1 - Towards Large Fine-grained Object Segmentation
  • Winner 2 - An Enhanced Mask R-CNN for Large Object Segmentation

Afternoon Session (January 8, UTC Time)

08:00 am - 08:10 am Introduction
08:10 am - 09:10 am Keynote Talk 1 (Andreas Geiger).
Neural Implicit Representations for 3D Vision
Video
09:10 am - 10:10 am Keynote Talk 2 (Weiwei Xu)
Data-driven 3D Content Creation
Video
10:10 am - 11:10 am Keynote Talk 3 (Chaohui Wang)
Occlusion Boundary & Pixel-Pair Occlusion Relationship Map
Video
11:10 am - 11:25 am Coffee Break.
11:25 am - 12:25 pm Contributed talk (3) - Cross-domain Image-based 3D Shape Retrieval
  • Winner 1 - Image-based 3D Shape Retrieval with Similarity-guided Label Modification
  • Winner 2 - Multi-view Based Joint Learning Network for 2D Image-based 3D Shape Retrieval
  • Winner 3 - 3D Shape Retrieval Solution Based On Multi-View Model


Industry Talk

Binqiang Zhao Multimedia Understanding and Recommendation for Better User Experiences and Production incentives Video


嘉宾

Andreas Geiger is professor at the University of Tübingen and group leader at the Max Planck Institute for Intelligent Systems. Prior to this, he was a visiting professor at ETH Zürich and a research scientist at MPI-IS. He studied at KIT, EPFL and MIT and received his PhD degree in 2013 from the KIT. His research interests are at the intersection of 3D reconstruction, motion estimation, scene understanding and sensory-motor control. He maintains the KITTI vision benchmark.


Tatsuya Harada is a Professor in the Research Center for Advanced Science and Technology at the University of Tokyo. His research interests center on visual recognition, machine learning, and intelligent robot. He received his Ph.D. from the University of Tokyo in 2001. He is also a team leader at RIKEN AIP and a vice director of Research Center for Medical Bigdata at National Institute of Informatics, Japan.


Jun-Yan Zhu is an Assistant Professor with The Robotics Institute in the School of Computer Science of Carnegie Mellon University. He also holds affiliated faculty appointments in the Computer Science Department and Machine Learning Department. Prior to joining CMU, he was a Research Scientist at Adobe Research and a postdoctoral researcher at MIT CSAIL. He obtained his Ph.D. from UC Berkeley and his B.E. from Tsinghua University. He studies computer vision, computer graphics, computational photography, and machine learning. He is the recipient of the Facebook Fellowship, ACM SIGGRAPH Outstanding Doctoral Dissertation Award, and UC Berkeley EECS David J. Sakrison Memorial Prize for outstanding doctoral research. His co-authored work has received the NVIDIA Pioneer Research Award, SIGGRAPH 2019 Real-time Live! Best of Show Award and Audience Choice Award, and The 100 Greatest Innovations of 2019 by Popular Science.


Rongfei Jia is a Staff Algorithm Expert at Tao Technology Department, Alibaba Group. He received his PhD degree from Beihang University. He is leading an algorithm team which is devoted in 3D AI algorithms and AR technologies to make it easier for online retail, especially for household furniture. His research interests include 3D object modeling, furniture scene understanding & synthesis, and recommender systems.


Weiwei Xu is currently a researcher at state key lab of CAD&CG in Zhejiang university. Before that, he was a Qianjiang Professor at Hangzhou Normal University and a researcher in Internet Graphics Group at Microsoft Research Asia from 2005 to 2012, and he was a post-doc researcher at Ritsmeikan university in Japan for more than one year. He received Ph.D. Degree in Computer Graphics from Zhejiang University, Hangzhou, and B.S. Degree and Master Degree in Computer Science from Hohai University in 1996 and 1999 respectively. He has published more than 80 papers on peer-review conference and journal papers, including around 30 papers on top-tier journals and conferences, such as ACM Transactions on Graphics, IEEE TVCG, CVPR and AAAI. He won the NSFC outstanding young research award on 2013, and second prize for natural sciences at Zhejiang province.


Chaohui Wang is Maître de Conférences (Associate Professor) at Université Paris-Est, Marne-la-Vallée, permanent researcher at LIGM Laboratory (UMR 8049), Université Gustave Eiffel, CNRS, ESIEE Paris, Ecole des Ponts, France. He received his PhD degree in Applied Mathematics at École Centrale Paris, France, under the supervision of Prof. Nikos Paragios. After that, he was postdoctoral researcher, working with Prof. Stefano Soatto at University of California, Los Angeles, USA, and then with Prof. Michael J. Black at Max Planck Institute for Intelligent Systems, Germany. He is a Senior Member of the IEEE. His research interests include computer vision, machine learning, and related problems.


Workshop 组织者

Huan Fu

Alibaba-inc

Rongfei Jia

Alibaba-inc

Lin Gao

ICT.CAS

Mingming Gong

University of Melbourne

Binqiang Zhao

Alibaba-inc

Steve Maybank

Birkbeck College, University of London

Dacheng Tao

The University of Sydney

合作伙伴