Brian Alderman | MCT, CEO / Founder of MicroTechPoint

1. 04 | Grouping and Aggregating Data Brian Alderman | MCT, CEO / Founder of MicroTechPoint Tobias Ternstrom | Microsoft SQL Server Program Manager

2. Course Topics

3. Aggregate functions GROUP BY and HAVING clauses Subqueries (self-contained, correlated, and EXISTS) Working with table functions Module Overview

4. Aggregate Functions

5. Common built-in aggregate functions Common Statistical Other • STDEV • STDEVP • VAR • VARP • SUM • MIN • MAX • AVG • COUNT • COUNT_BIG • CHECKSUM_AGG • GROUPING • GROUPING_ID

6. Working with aggregatefunctions • Aggregate functions: • Return a scalar value (with no column name) • Ignore NULLs except in COUNT(*) • Can be used in • SELECT, HAVING, and ORDER BY clauses • Frequently used with GROUP BY clause SELECT COUNT (DISTINCTSalesOrderID) ASUniqueOrders, AVG(UnitPrice) ASAvg_UnitPrice, MIN(OrderQty)ASMin_OrderQty, MAX(LineTotal) ASMax_LineTotal FROMSales.SalesOrderDetail; UniqueOrdersAvg_UnitPriceMin_OrderQtyMax_LineTotal ------------- ------------ ------------ ------------- 31465 465.0934 1 27893.619000

7. Using DISTINCT with aggregate functions • Use DISTINCT with aggregate functions to summarize only unique values • DISTINCT aggregates eliminate duplicate values, not rows (unlike SELECT DISTINCT) • Compare (with partial results): SELECTSalesPersonID, YEAR(OrderDate) ASOrderYear, COUNT(CustomerID) ASAll_Custs, COUNT(DISTINCTCustomerID) ASUnique_Custs FROMSales.SalesOrderHeader GROUP BY SalesPersonID, YEAR(OrderDate); SalesPersonIDOrderYearAll_CustsUnique_custs ----------- ----------- ----------- ------------ 289 2006 84 48 281 2008 52 27 285 2007 9 8 277 2006 140 57

8. Using the GROUP BY clause • GROUP BY creates groups for output rows, according to unique combination of values specified in the GROUP BY clause • GROUP BY calculates a summary value for aggregate functions in subsequent phases • Detail rows are “lost” after GROUP BY clause is processed SELECT <select_list> FROM <table_source> WHERE <search_condition> GROUP BY <group_by_list>; SELECTSalesPersonID, COUNT(*) ASCnt FROMSales.SalesOrderHeader GROUP BY SalesPersonID;

9. Demo Using Aggregate functions

10. GROUP BY and HAVING

11. GROUP BY and logical order of operations • HAVING, SELECT, and ORDER BY must return a single value per group • All columns in SELECT, HAVING, and ORDER BY must appear in GROUP BY clause or be inputs to aggregate expressions • If a query uses GROUP BY, all subsequent phases operate on the groups, not source rows

12. Using GROUP BY with aggregate functions • Aggregate functions are commonly used in SELECT clause, summarize per group: • Aggregate functions may refer to any columns, not just those in GROUP BY clause SELECTCustomerID, COUNT(*) AScnt FROMSales.SalesOrderHeader GROUP BY CustomerID; SELECT productid, MAX(OrderQty) AS largest_order FROMSales.SalesOrderDetail GROUP BY productid;

13. Filtering grouped data using HAVING Clause • HAVING clause provides a search condition that each group must satisfy • HAVING clause is processed after GROUP BY SELECTCustomerID, COUNT(*) ASCount_Orders FROMSales.SalesOrderHeader GROUP BY CustomerID HAVINGCOUNT(*) > 10;

14. Compare HAVING to WHERE clauses • WHERE filters rows before groups created • Controls which rows are placed into groups • HAVING filters groups • Controls which groups are passed to next logical phase • Using a COUNT(*) expression in HAVING clause is useful to solve common business problems: • Show only customers that have placed more than one order: • Show only products that appear on 10 or more orders: SELECTCust.Customerid, COUNT(*) AScnt FROMSales.CustomerASCust JOINSales.SalesOrderHeaderASOrdONCust.CustomerID = ORD.CustomerID GROUP BY Cust.CustomerID HAVINGCOUNT(*) > 1; SELECTProd.ProductID, COUNT(*) AScnt FROMProduction.ProductAS Prod JOINSales.SalesOrderDetailASOrdONProd.ProductID = Ord.ProductID GROUP BY Prod.ProductID HAVINGCOUNT(*) >= 10;

15. Demo Using GROUP BY and HAVING

16. Subqueries

17. Working with subqueries • Subqueries are nested queries or queries within queries • Results from inner query are passed to outer query • Inner query acts like an expression from perspective of outer query • Subqueries can be self-contained or correlated • Self-contained subqueries have no dependency on outer query • Correlated subqueries depend on values from outer query • Subqueries can be scalar, multi-valued, or table-valued

18. Writing scalar subqueries • Scalar subquery returns single value to outer query • Can be used anywhere single-valued expression can be used: SELECT, WHERE, etc. • If inner query returns an empty set, result is converted to NULL • Construction of outer query determines whether inner query must return a single value SELECTSalesOrderID, ProductID, UnitPrice, OrderQty FROMSales.SalesOrderDetail WHERESalesOrderID = (SELECTMAX(SalesOrderID) ASLastOrder FROMSales.SalesOrderHeader);

19. Writing multi-valued subqueries • Multi-valued subquery returns multiple values as a single column set to the outer query • Used with IN predicate • If any value in the subquery result matches IN predicate expression, the predicate returns TRUE • May also be expressed as a JOIN (test both for performance) SELECTCustomerID, SalesOrderId,TerritoryID FROMSales.SalesorderHeader WHERECustomerIDIN ( SELECTCustomerID FROMSales.Customer WHERETerritoryID = 10);

20. Writing queries using EXISTS with subqueries • The keyword EXISTS does not follow a column name or other expression. • The SELECT list of a subquery introduced by EXISTS typically only uses an asterisk (*). SELECTCustomerID, PersonID FROMSales.CustomerASCust WHERE EXISTS ( SELECT * FROMSales.SalesOrderHeaderASOrd WHERE Cust.CustomerID = Ord.CustomerID); SELECTCustomerID, PersonID FROMSales.CustomerASCust WHERE NOT EXISTS ( SELECT * FROMSales.SalesOrderHeaderASOrd WHERECust.CustomerID = Ord.CustomerID);

21. Demo Using subqueries

22. Table Functions

23. Creating simple views • Views are saved queries created in a database by administrators and developers • Views are defined with a single SELECT statement • ORDER BY is not permitted in a view definition without the use of TOP, OFFSET/FETCH, or FOR XML • To sort the output, use ORDER BY in the outer query • View creation supports additional options beyond the scope of this class CREATEVIEWHumanResources.EmployeeList AS SELECTBusinessEntityID, JobTitle,HireDate,VacationHours FROMHumanResources.Employee; SELECT * FROMHumanResources.EmployeeList

24. Creating simple inline table-valued functions • Table-valued functions are created by administrators and developers • Create and name function and optional parameters with CREATE FUNCTION • Declare return type as TABLE • Define inline SELECT statement following RETURN CREATE FUNCTION Sales.fn_LineTotal (@SalesOrderIDINT) RETURNS TABLE AS RETURN SELECTSalesOrderID, CAST((OrderQty * UnitPrice * (1 - SpecialOfferID)) AS DECIMAL(8, 2)) AS LineTotal FROM Sales.SalesOrderDetail WHERESalesOrderID = @SalesOrderID ;

25. Writing queries with derived tables • Derived tables are named query expressions created within an outer SELECT statement • Not stored in database – represents a virtual relational table • When processed, unpacked into query against underlying referenced objects • Allow you to write more modular queries • Scope of a derived table is the query in which it is defined SELECT<column_list> FROM( <derived_table_definition> )AS<derived_table_alias>;

26. Guidelines for derivedtables Derived Tables Must Derived Tables May • Have an alias • Have names for all columns • Have unique names for all columns • Not use an ORDER BY clause (without TOP or OFFSET/FETCH) • Not be referred to multiple times in the same query • Use internal or external aliases for columns • Refer to parameters and/or variables • Be nested within other derived tables

27. Passing arguments to derived tables • Derived tables may refer to arguments • Arguments may be: • Variables declared in the same batch as the SELECT statement • Parameters passed into a table-valued function or stored procedure DECLARE @emp_id INT= 9; SELECT orderyear,COUNT(DISTINCT custid)AS cust_count FROM ( SELECTYEAR(orderdate)AS orderyear, custid FROM Sales.Orders WHERE empid=@emp_id )AS derived_year GROUPBY orderyear;

28. Creating queries with common table expressions • Use WITH clause to create a CTE: • Define the table expression in WITH clause • Reference the CTE in the outer query • Assign column aliases (inline or external) • Pass arguments if desired WITHCTE_yearAS ( SELECTYEAR(OrderDate) ASOrderYear, customerID FROMSales.SalesOrderHeader ) SELECTorderyear, COUNT(DISTINCT CustomerID) ASCustCount FROM CTE_year GROUP BY OrderYear;

29. Demo Table functions

30. Summary • Aggregate functions are used in SELECT, HAVING, and ORDER By clauses, but are most frequently used with the GROUP BY clause and returns a scalar value • Common built-in aggregate functions include Common Statistical Other • STDEV • STDEVP • VAR • VARP • SUM • MIN • MAX • AVG • COUNT • COUNT_BIG • CHECKSUM_AGG • GROUPING • GROUPING_ID

31. Summary • Use DISTINCT with aggregate functions to only summarize the unique values as it will eliminate duplicate values, not rows • GROUP BY creates groups for output rows, according to unique combination of values specified in the GROUP BY clause. GROUP BY also calculates a summary value for aggregate functions in subsequent phases • HAVING clause provides a search condition that each group must satisfy and is processed after the GROUP BY clause

32. Summary • Subqueries are nested queries or queries within queries where the results from inner query are passed to the outer query • Type of subqueries include • Scalar subqueries • Multi-valued subqueries • Subqueries with the EXISTS clause

33. Summary Views are named tables expressions with definitions stored in a database that can be referenced in a SELECT statement just like a table Views are defined with a single SELECT statement and then saved in the database as queries Table-valued functions are created with the CREATE FUNCTION. They contain a RETURN type of table Derived tables allow you to write more modular queries as named query expressions that are created within an outer SELECT statement. They represent a virtual relational table so are not stored in the database CTEs are similar to derived tables in scope and naming requirements but unlike derived tables, CTEs support multiple definitions, multiple references, and recursion

34. Course Topics