2023-11-07-Gartner-Gartner++AI+和软件工程团队必须进行的3次紧急对话-英-46页_46页_1mb

Gartner Webinars Summary

Core Content

Gartner Webinars focus on the integration of AI and Software Engineering (SWE) within organizations, emphasizing the need for collaboration, shared data pipelines, and aligned operational processes. The webinars highlight the growing importance of AI in enterprise systems and the necessity for both AI and SWE teams to work together to enhance performance, reduce risk, and improve innovation.

Main Points

1. The Urgency of AI and SWE Collaboration

• Complexity, Risk, Innovation, Speed, Efficiency, and Quality are key drivers for the urgent need of collaboration between AI and SWE teams.
• Mature AI organizations involve software engineers more deeply in the AI use-case lifecycle, ensuring better alignment and outcomes.
• AI and SWE projects are not just about algorithms; they involve systems, data, and operational considerations.

2. Shared Data Pipeline Development

• A shared data pipeline is essential for AI and SWE teams to avoid duplication and improve efficiency.
• Key components include data extraction, transformation, mining, load, data cleaning, PII services, data fusion, metadata, validation, and feature engineering.
• Metadata and API access are crucial for scaling AI systems across hundreds or thousands of models.
• Collaboration on synthetic data and Generative AI can create new economies of scale.

3. Operational Responsibilities and Shared Duties

• AI teams are primarily responsible for model development, training, and governance, while SWE teams handle application development, deployment, and operations.
• ModelOps and DevOps overlap requires clear roles and responsibilities to avoid operational gaps.
• Shared operational pipelines should include model testing, integration, deployment, and monitoring, with CI/CD processes that support both AI and software development.

4. Skill Exchange and Literacy

• AI skills required by software engineers include AI techniques, data literacy, model testing, and ethics.
• Software engineering skills needed by AI engineers include systems development principles, version control, deployment best practices, and performance optimization.
• Cross-training is recommended to improve communication and collaboration between teams.
• Data literacy programs for software engineers and AI literacy for SWE teams are essential.

5. Organizational and Process Recommendations

• Clear goals and roles are necessary to define the collaboration and reduce duplication.
• Shared design patterns and knowledge graphs can enhance systems design and data enrichment.
• Formal collaboration is increasingly common, with 80% of organizations having formal or standard operating procedures.
• Unified XOps pipelines (e.g., ModelOps and DevOps) are recommended for seamless integration.

Key Information

Data Pipeline Challenges

• Data challenges are among the top 3 barriers to AI implementation.
• Unstructured data dominates enterprise data, making data pipeline development more complex.
• The average enterprise has 347.56TB of data, which is growing rapidly.

Roles in AI Use-Case Lifecycle

• Mature AI organizations involve more software engineers across the AI lifecycle.
• Roles involved include data scientists, software engineers, and operational teams, with responsibilities varying by maturity level.

Skills Required

• Software Engineers need to understand AI techniques, data literacy, and model development.
• AI Engineers need to understand software engineering principles, version control, and deployment best practices.

Training and Knowledge Transfer

• Training programs should be role-specific, covering areas like regression testing, model governance, and data literacy.
• Knowledge exchange programs with vendors and suppliers can support skill development and collaboration.

Deliverables

• Pipeline flow diagrams and RACI models for shared data pipelines.
• A first draft of an operating model and a roadmap for AI and SWE operations.
• Training programs with prioritization by role and knowledge exchange initiatives.

Recommendations

• Establish clear operating models with defined goals and responsibilities.
• Reduce duplication by creating a shared pool of design patterns.
• Adopt knowledge graphs to support data enrichment and integration.
• Focus on model testing, integration, and monitoring to ensure operational alignment.
• Cross-train teams and develop data and AI literacy programs to enhance collaboration.

Conclusion

Gartner emphasizes the critical need for collaboration between AI and Software Engineering teams to achieve successful AI implementation. By focusing on shared data pipelines, unified operational processes, and skill exchange, organizations can improve efficiency, reduce risk, and drive innovation.