2017年-数据局_斯坦福年度AI报告：人工智能全面逼近人类能力_101页_7mb

AI Index 2017 Annual Report Summary

Core Content

The AI Index 2017 Annual Report is an open, not-for-profit project launched by the One Hundred Year Study on AI at Stanford University. It aims to track AI activity and progress through various metrics and provide a data-driven basis for informed discussions about AI. The report includes data from academia, industry, open-source software, and public interest, along with technical performance benchmarks and derivative measures.

Main Sections and Key Points

Volume of Activity

• Academia:
  • The number of AI-related papers published in the Scopus database has increased more than 9x since 1996.
  • Course enrollment in introductory AI and ML courses at Stanford has grown 11x since 1996, indicating a surge in academic interest.
  • Conference attendance has shifted from symbolic reasoning to machine learning and deep learning.

Industry

• AI-Related Startups:
  • The number of active US AI startups has increased 14x since 2000.
• Startup Funding:
  • Annual VC investment into US AI startups has grown 6x since 2000.
• Job Openings:
  • The share of US jobs requiring AI skills has grown 4.5x since 2013.
  • AI job openings on Monster.com show a breakdown by specific skills, highlighting the growing demand for AI expertise.

Open Source Software

• GitHub Stars:
  • TensorFlow and Scikit-Learn have seen significant interest, with stars indicating developer engagement and usage.

Public Interest

• Media Sentiment:
  • The percentage of articles containing the term "Artificial Intelligence" and classified as positive or negative is analyzed, reflecting public perception trends.

Technical Performance

• Vision:

  • Object detection error rates have dropped from 28.5% in 2010 to below 2.5%.
  • Visual Question Answering (VQA) performance is explored, though the VQA 1.0 dataset is outdated.

• Natural Language Understanding:

  • Parsing accuracy has improved significantly.
  • Machine translation between English and German has reached high levels of performance.
  • Question answering systems, such as those tested on SQuAD v1.1, have achieved strong results.
  • Speech recognition systems have approached human-level performance in specific domains.

• Theorem Proving:

  • The tractability of theorem proving problems has increased, indicating improved capabilities of AI systems.

• SAT Solving:

  • Competitive SAT solvers have shown strong performance on industry-applicable problems.

Derivative Measures

• Academia-Industry Dynamics:

  • The report explores the relationship between academic and industry activity, showing that while academic activity initially drove progress, industry investment became a major driver around 2013.
  • Metrics are normalized to 2000 for comparison.

• AI Vibrancy Index:

  • A composite index combining normalized publishing, enrollment, and investment metrics to quantify the overall activity and progress in AI.
  • The index shows that academia has caught up with industry in terms of activity and progress.

Towards Human-Level Performance?

• Milestones:
  • AI systems have achieved human-level performance in several areas:
    • Othello (1989): BILLL and Logistello beat top human players.
    • Checkers (1995): Chinook defeated the world champion.
    • Chess (1997): DeepBlue beat Gary Kasparov.
    • Jeopardy! (2011): IBM Watson won against human champions.
    • Atari Games (2015): DeepMind's system achieved human-level performance in many games.
    • Go (2016): AlphaGo beat Lee Sedol and later AlphaGo Zero beat the original AlphaGo system.
    • Skin Cancer Classification (2017): An AI system matched dermatologists in diagnosing skin cancer.
    • Speech Recognition (2017): Microsoft and IBM reached human-parity in specific domains.
    • Poker (2017): Libratus and DeepStack demonstrated strong performance against human players.
    • Pac-Man (2017): Maluuba's AI system achieved the maximum score in the game.

What's Missing?

• Technical Areas:
  • Some areas lack standardized benchmarks, such as dialogue systems, planning, and continuous control in robotics.
  • Commonsense reasoning and other complex tasks are underrepresented.
• International Coverage:
  • The report is heavily US-centric, with limited data on AI activity in other countries, such as China.
• Diversity & Inclusion:
  • The report lacks demographic data and does not measure who is involved in AI research and deployment.
• Government and Corporate Investment:
  • There is no data on AI R&D investments by governments and corporations.

Call to Action

• The AI Index encourages broader community involvement in data collection, analysis, and metric development.
• It invites contributions and ideas to address current limitations and expand the scope of the report.

Conclusion

The AI Index 2017 Annual Report provides a comprehensive overview of AI activity and progress, highlighting both the rapid growth and the areas that still need deeper exploration. It serves as a foundation for data-driven discussions and a call to action for further research and collaboration in the AI field.