首页 > 代码库 > Spark2 DataFrame数据框常用操作(七)之统计指标:mean均值,variance方差,stddev标准差,corr(Pearson相关系数),skewness偏度,kurtosis峰度

Spark2 DataFrame数据框常用操作(七)之统计指标:mean均值,variance方差,stddev标准差,corr(Pearson相关系数),skewness偏度,kurtosis峰度

val df4=spark.sql("SELECT mean(age),variance(age),stddev(age),corr(age,yearsmarried),skewness(age),kurtosis(age) FROM Affairs")

df4.show
+--------+------------------+------------------+-----------------------+-----------------+--------------------+
|avg(age)|     var_samp(age)|  stddev_samp(age)|corr(age, yearsmarried)|    skewness(age)|       kurtosis(age)|
+--------+------------------+------------------+-----------------------+-----------------+--------------------+
|    34.0|173.33333333333334|13.165611772087667|     0.7456766124552038|0.965388004190285|-0.43417159763313595|
+--------+------------------+------------------+-----------------------+-----------------+--------------------+

 

Spark2 DataFrame数据框常用操作(七)之统计指标:mean均值,variance方差,stddev标准差,corr(Pearson相关系数),skewness偏度,kurtosis峰度