2017年-数据局_斯坦福：AI百年报告2017_101页_5mb

AI Index 2017 Annual Report Summary

Core Content

The AI Index 2017 Annual Report is an open, not-for-profit initiative launched by the One Hundred Year Study on AI at Stanford University (Al100). It aims to track AI activity and progress through various metrics, enabling informed discussions about the field's development. The report is structured into data sections and discussion sections, highlighting both quantitative trends and qualitative expert opinions.

Main Sections & Key Points

Volume of Activity

• Academia:
  • The number of AI-related papers published annually has increased by more than 9x since 1996.
  • Course enrollment in introductory AI and Machine Learning (ML) courses at Stanford has grown 11x since 1996.
  • Conference attendance has shifted from symbolic reasoning to machine learning and deep learning, indicating a change in research focus.
• Industry:
  • The number of active US AI startups has increased 14x since 2000.
  • Annual VC investment in AI startups has grown 6x since 2000.
  • AI job openings have increased 4.5x since 2013 in the US, though Canada and the UK are smaller in comparison.
• Robot Imports:
  • There has been a significant increase in the import of industrial robots into North America and globally.

Open Source Software

• GitHub Stars:
  • The popularity of AI and ML software packages like TensorFlow and Scikit-Learn is reflected in their GitHub star counts.
  • Developers use stars to indicate interest in and usage of software, which can serve as a metric for community engagement.

Public Interest

• Media Sentiment:
  • The percentage of media articles referencing AI and classified as positive or negative is analyzed, indicating public perception trends.

Technical Performance

• Vision:
  • Object detection error rates in the LSVRC competition have dropped from 28.5% in 2010 to below 2.5%.
• Natural Language Understanding:
  • Parsing: AI systems have improved in determining the syntactic structure of sentences.
  • Machine Translation: Performance in translating news between English and German has improved significantly.
  • Question Answering: AI systems have reached human-level performance in the SQuAD v1.1 dataset.
  • Speech Recognition: Both Microsoft and IBM achieved human-parity performance in the Switchboard domain.
• Theorem Proving & SAT Solving:
  • The tractability of theorem proving problems has improved, with more problems being solvable by state-of-the-art solvers.
  • Competitive SAT solvers have solved a growing percentage of industry-applicable problems.

Milestones Toward Human-Level Performance

• Othello (1989, 1997): AI systems have consistently beaten human champions in this game.
• Checkers (1995): Chinook defeated the world champion, marking a significant milestone.
• Chess (1997): IBM's DeepBlue beat Gary Kasparov, and modern systems can now play at the grandmaster level.
• Jeopardy! (2011): IBM Watson won the quiz show, showcasing AI's ability to handle complex language tasks.
• Atari Games (2015): Google DeepMind's AI system achieved human-level performance in most games but struggled with more complex ones.
• Go (2016): AlphaGo defeated Lee Sedol and later AlphaGo Zero beat the original AlphaGo system.
• Skin Cancer Classification (2017): An AI system matched the performance of dermatologists in diagnosing skin cancer.
• Poker (2017): Programs like Libratus and DeepStack demonstrated strong performance against human professionals.
• Ms. Pac-Man (2017): An AI system learned to achieve the game's maximum score.

Derivative Measures

• Academia-Industry Dynamics:
  • The report examines the relationship between academic and industry activities.
  • Academic activity initially drove progress, but industry investment became a major driver by 2013.
  • Academia has since caught up with industry's enthusiasm.
• AI Vibrancy Index:
  • Combines normalized metrics from publishing, enrollment, and VC investment to quantify the field's vitality.
  • The index reflects the combined growth of these areas, indicating a dynamic and evolving AI landscape.

What's Missing?

• Technical Performance Gaps:
  • Areas like dialogue systems, planning, and continuous control in robotics lack clear benchmarks.
  • Common sense reasoning and standardized testing are difficult to measure.
• International Coverage:
  • The report is heavily US-centric and lacks data on global AI activity, especially from countries like China.
• Demographics & Inclusion:
  • The report does not include demographic breakdowns of AI researchers and practitioners.
  • There is a need to understand who is involved in AI development and how diverse the field is.

Call to Action

• The AI Index is a call for participation and collaboration.
• The community is encouraged to contribute data, analyze trends, and suggest new metrics.
• Future reports aim to address current limitations by including more international data and diverse perspectives.

Conclusion

The AI Index 2017 Annual Report provides a comprehensive overview of AI's growth in academia and industry, highlighting significant technical advancements and shifts in research focus. It also emphasizes the importance of continued data collection and analysis to ensure a more accurate and inclusive understanding of AI's progress and societal impact.