2018年-数据局_大数据文摘：2017AI趋势报告_101页_7mb

AI Index 2017 Annual Report Summary

Core Content

The AI Index 2017 Annual Report is an open, not-for-profit project launched by the One Hundred Year Study on AI at Stanford University (Al100). It aims to track activity and progress in AI through data-driven analysis, providing a comprehensive overview of AI's growth and impact across various domains.

Main Sections

Volume of Activity

• Academia:

  • The number of AI-related papers published in the Scopus database has increased by over 9x since 1996.
  • Annual AI paper publication in Computer Science grew by more than 9x, while general Computer Science papers grew by 6x.
  • Stanford's introductory AI and ML course enrollments increased by 11x since 1996, with a dip in 2016 due to administrative reasons.

• Industry:

  • The number of active US AI startups increased by 14x since 2000.
  • Annual VC funding for AI startups grew by 6x since 2000.
  • Job openings requiring AI skills on Indeed.com increased by 4.5x since 2013, though the US still dominates the market compared to Canada and the UK.

• Robot Imports:

  • Industrial robot shipments to North America and globally have seen significant growth.

• Open Source Software:

  • GitHub stars for AI/ML software packages such as TensorFlow and Scikit-Learn indicate developer interest and usage.
  • Forks of GitHub repositories also show similar growth trends.

Technical Performance

• Vision:

  • Error rates in image labeling tasks have dropped from 28.5% in 2010 to below 2.5% in 2016.
  • Visual Question Answering (VQA) systems have made progress, though the VQA 1.0 dataset is outdated.

• Natural Language Understanding:

  • Constituency parsing performance has improved significantly.
  • Machine translation between English and German has reached high accuracy.
  • Question Answering systems, such as those evaluated on SQuAD v1.1, have improved.
  • Speech recognition systems have achieved performance close to human parity in the Switchboard domain.

• Theorem Proving and SAT Solving:

  • The tractability of theorem proving problems has improved, with more problems solvable by state-of-the-art systems.
  • SAT solvers have shown strong performance on industry-applicable problems.

Derivative Measures

• Academia-Industry Dynamics:

  • Academic activity (papers and course enrollments) initially drove AI progress.
  • From 2013 onward, industry investment became the primary driver.
  • Academia has since caught up with industry in terms of activity.

• AI Vibrancy Index:

  • A composite index combining normalized metrics from publishing, enrollment, and VC investment to quantify the field's vitality.
  • The index shows a steady increase in AI activity, with academia and industry contributing similarly since 2013.

Towards Human-Level Performance?

• AI systems have achieved human-level performance in several specific tasks:
  • Othello: BILLL and Logistello beat top human players.
  • Checkers: Chinook defeated the world champion in 1995.
  • Chess: IBM's DeepBlue beat Gary Kasparov in 1997.
  • Jeopardy!: IBM Watson won the quiz show.
  • Atari Games: DeepMind's AI achieved human-level performance in many games.
  • ImageNet Object Detection: Error rates dropped to below 3%, close to human performance.
  • Go: AlphaGo beat Lee Sedol and later AlphaGo Zero defeated the original AlphaGo system.
  • Skin Cancer Classification: An AI system matched the performance of dermatologists.
  • Speech Recognition: Microsoft and IBM achieved human-parity performance in Switchboard.
  • Poker: Libratus and DeepStack outperformed human professionals.

What's Missing?

• The report highlights several limitations:
  • Technical Areas: Some important areas like dialogue systems, planning, and commonsense reasoning lack standardized benchmarks.
  • International Coverage: The report is US-centric, with limited data on global AI activity, especially in countries like China.
  • Demographics: There is no breakdown of AI researchers or participants by gender or ethnicity.
  • Government and Corporate Investment: Data on AI R&D investments by governments and corporations is missing.
  • Public Interest: The report does not fully capture public sentiment or media coverage of AI.

Expert Forum and Call to Action

• The report includes subjective commentary from AI experts to provide context and interpretation to the data.
• It encourages broader community participation in data collection, analysis, and the development of new metrics to better measure AI progress and its societal impact.

Conclusion

The AI Index 2017 Annual Report provides a snapshot of AI's rapid growth and progress, highlighting key areas in academia, industry, and open-source software. It underscores the importance of data-driven discussions and invites further contributions to ensure a more comprehensive and inclusive understanding of AI's development and impact.