【T112017-数据工程和技术分会场】TiDB+as+an+HTAP+Database_26页_2mb

TiDB as an HTAP Database Summary

Core Content

TiDB is a NewSQL database designed to support both OLTP (Online Transaction Processing) and OLAP (Online Analytical Processing) in a unified system, making it an HTAP (Hybrid Transactional/Analytical Processing) database. It is built to provide horizontal scalability, high availability, and 24/7 availability, even during datacenter outages. TiDB is open source, compatible with MySQL, and aims to simplify data management by eliminating the need for separate OLTP and OLAP systems.

Main Points

• Why a New Database?
  Traditional databases struggle with handling both transactional and analytical workloads efficiently. TiDB addresses this by combining the strengths of relational databases and NoSQL systems, enabling real-time analysis and transaction support without compromising performance or consistency.

• Brief History
  TiDB evolved from the need to solve scalability and availability issues in traditional RDBMS and NoSQL systems. It was created as a NewSQL database that supports ACID transactions at scale while also providing SQL support for analytical queries.

• NewSQL Features

  • Horizontal Scalability: Supports distributed architecture for handling large data volumes.
  • High Availability: Ensures data is always accessible, even in the event of failures.
  • ACID Transactions: Maintains data consistency and integrity across distributed systems.
  • SQL at Scale: Enables complex queries and analytical processing while maintaining transactional guarantees.

• HTAP Capabilities
  TiDB combines OLTP and OLAP capabilities, allowing for real-time analysis and ACID transactions in the same system. This eliminates the need for separate systems for transactional and analytical workloads.

• Architecture Overview

  • Region-based Data Storage: Data is organized into regions, each managed by a Raft group.
  • Stateless SQL Layer: The SQL layer is stateless and can be scaled independently.
  • TiKV as Storage Engine: TiKV provides distributed storage with strong consistency and high availability.
  • PD (Placement Driver): Manages the cluster, including load balancing and data placement.

Key Information

• Scalability: TiDB is built with scalability as a core feature, allowing it to handle large-scale data and high concurrency.
• Compatibility: It is compatible with MySQL in most cases, making it easier for developers to migrate and use.
• HTAP: TiDB supports both transactional and analytical processing, reducing the need for data duplication and ETL processes.
• Query Optimization: TiDB uses a Cost-Based Optimizer (CBO) to choose the most efficient access paths, including:
  • Predicate Pushdown
  • Column Pruning
  • Eager Aggregation
  • Subquery to Join Conversion
  • Index Selection
  • Join Operator Selection (Hash, Index Lookup, Sort-Merge)
• Ecosystem: TiDB integrates with the Spark ecosystem and supports complex calculation pushdown, index support, and mixed storage engines (columnar/row-based), along with heterogeneous computing (CPU/GPU/FPGA) for enhanced performance.

Conclusion

TiDB is a distributed, horizontally scalable, and highly available NewSQL database that supports HTAP. It is designed to handle both transactional and analytical workloads in real time, offering SQL compatibility, advanced query optimization, and a robust ecosystem for data processing and analytics. This makes TiDB a powerful solution for modern applications that require both high throughput and low latency.