The functions SUM, AVG, COUNT, MIN, and MAX are well-known aggregate functions that we use every day. They are to compute/summarize the multiple rows by grouping them and provide a single summary value.
However, when it comes to analysis, the aggregate functions carry out twofold responsibility: The same aggregate function which computes on each row can also help in computing based on the range of rows or partitions.
Considering the “EMP” data, the following aggregations/analysis can be done based on Salary.
1) Identify the total salary costing to the company.
2) Identify the total salary budget for each department.
3) Identify the total salary budget for each role in each department.
This means, in the sales department, how much salary is being paid to Salesmen and to other teams within the department.
Point (1) can be achieved with a simple aggregation however the other two are not limited to one result value and operate on a certain range or partition or window where the input rows are ordered and grouped using flexible conditions like data window/range of rows expressed through an OVER() clause. The range or partition in the above scenario is “department” for the 2nd point and “Job” for the 3rd one.
Here is the data for an example.
Aggregate functions with OVER clause:
SELECT DISTINCT [JOB]
,a.[DEPTNO],
b.[DNAME],
b.LOC,
SUM(SAL) OVER(PARTITION BY a.DeptNo) DepartmentwiseSalaryCTC,
SUM(SAL) OVER(PARTITION BY a.Job) JobwiseSalaryCTC
FROM [TestDB1].[dbo].[EMP] a
JOIN [TestDB1].[dbo].[DEPT] b ON a.DeptNo = b.DeptNo
If you look at the summary, employees salary cost to company based on department and job is retrieved. This way, we can use all the aggregate functions with the OVER clause to analyze the data in the most possible ways.
SELECT DISTINCT [JOB], a.[DEPTNO], b.[DNAME],
SUM(SAL) OVER(PARTITION BY a.DeptNo) DepartmentSalaryCTC,
SUM(SAL) OVER(PARTITION BY a.Job) JobSalaryCTC,
AVG(SAL) OVER(PARTITION BY a.DeptNo) AVGSalaryPerDept,
AVG(SAL) OVER(PARTITION BY a.Job) AVGSalaryPerRole,
MAX(SAL) OVER(PARTITION BY a.DeptNo) MaxSalaryInEachDept,
MAX(SAL) OVER(PARTITION BY a.Job) MaxSalaryInEachRole,
MIN(SAL) OVER(PARTITION BY a.DeptNo) MinSalaryInEachDept,
MIN(SAL) OVER(PARTITION BY a.Job) MinSalaryInEachRole
FROM [TestDB1].[dbo].[EMP] a
JOIN [TestDB1].[dbo].[DEPT] b ON a.DeptNo = b.DeptNo
Please note that the syntax or the way of analysis is the same in any RDBMS and Big Data technologies like Hive and Impala.
Big Data and SQL 
6 comments