2017年度人工智能指数报告（英文）_101页-7mb

AI Index 2017 Annual Report Summary

Core Content

The AI Index 2017 Annual Report is an open, not-for-profit initiative by the One Hundred Year Study on AI at Stanford University. It aims to track AI activity and progress through various metrics, offering a data-driven perspective on the field's evolution. The report is structured into sections covering Volume of Activity, Technical Performance, Derivative Measures, and Towards Human-Level Performance, with a focus on academic, industrial, and public interest data.

Main Sections & Key Points

Volume of Activity

• Academia:
  • The number of AI-related papers published annually has increased by more than 9x since 1996.
  • Stanford University's introductory AI and ML course enrollments have grown by 11x since 1996.
  • AI-related job openings in the US have increased by 4.5x since 2013, with AI skills becoming more prevalent in the job market.
• Industry:
  • The number of active US AI startups has increased by 14x since 2000.
  • Annual VC investment in US AI startups has grown by 6x since 2000.
• Robot Imports:
  • The number of industrial robot units shipped to North America and globally has seen significant growth.
• Open Source Software:
  • GitHub stars for AI/ML libraries like TensorFlow and Scikit-Learn indicate developer interest and usage.
  • The AI Index highlights the importance of tracking such metrics to understand community engagement.

Technical Performance

• Vision:
  • Object detection error rates in the LSVRC competition dropped from 28.5% in 2010 to below 2.5% in 2016.
  • Visual Question Answering (VQA) performance has improved, though the VQA 1.0 dataset is now outdated.
• Natural Language Understanding:
  • Parsing performance on the Penn Treebank WSJ dataset has improved significantly.
  • Machine translation between English and German has reached near-human levels in the WMT competition.
  • Question Answering systems like SQuAD v1.1 have shown strong performance.
  • Speech recognition systems have achieved human-parity in the Switchboard domain.
• Theorem Proving & SAT Solving:
  • The tractability of theorem proving problems has increased, indicating better performance of AI systems.
  • Competitive SAT solvers have shown strong performance on industry-applicable problems.

Derivative Measures

• Academia-Industry Dynamics:
  • Academic activity (papers and course enrollment) initially drove progress, but by 2013, industry investment (VC funding) became the primary driver.
  • Academia has since caught up with industry activity.
• AI Vibrancy Index:
  • This index combines normalized metrics from academia (papers, course enrollment) and industry (VC investment) to quantify the field's liveliness.
  • It provides a composite view of AI's overall growth and activity.

Towards Human-Level Performance

• The report outlines several milestones where AI systems have matched or exceeded human performance:
  • Othello (1989, 1997): AI systems have beaten top human players.
  • Checkers (1995): Chinook defeated the world champion.
  • Chess (1997): IBM's DeepBlue beat Gary Kasparov.
  • Jeopardy! (2011): IBM Watson won the game show.
  • Atari Games (2015): AI systems achieved human-level performance in most games.
  • Go (2016): AlphaGo beat Lee Sedol and later AlphaGo Zero defeated the original AlphaGo.
  • Skin Cancer Classification (2017): AI outperformed dermatologists in diagnosing skin cancer.
  • Speech Recognition (2017): Microsoft and IBM reached human-parity in speech recognition.
  • Poker (2017): Programs like Libratus and DeepStack demonstrated superior performance against human players.
  • Pac-Man (2017): An AI system learned to achieve maximum score in the game.

Limitations & Missing Areas

• The report acknowledges several limitations:
  • It is US-centric, lacking international data, especially from regions like China.
  • It does not include demographic breakdowns or government/corporate investment in AI R&D.
  • Some areas, such as commonsense reasoning, dialogue systems, and recommender systems, lack standardized benchmarks.
  • The report does not cover AI's societal impact or diversity in AI research and deployment.

Call to Action

• The AI Index is a collaborative effort and invites the broader community to contribute data, analyze trends, and suggest new metrics for tracking AI progress.
• It encourages participation in the Expert Forum and invites individuals to get involved in shaping future reports.

Conclusion

The AI Index 2017 Annual Report provides a comprehensive overview of AI's growth and achievements, emphasizing the importance of data in understanding and guiding the field. It highlights the need for more diverse and inclusive metrics, as well as better international coverage to ensure a balanced view of AI's global impact.