가야대학교 분성도서관
주메뉴
자료검색
- Home
- 상세정보
상세정보
부가기능
Hands-on big data analytics with PySpark : analyze large datasets and discover techniques for testing, immunizing, and parallelizing Spark jobs /
상세 프로파일
| 자료유형 | E-Book |
|---|---|
| 개인저자 | Lai, Rudy, author. Potaczek, Bart흢omiej, author. |
| 서명/저자사항 | Hands-on big data analytics with PySpark :analyze large datasets and discover techniques for testing, immunizing, and parallelizing Spark jobs /Rudy Lai, Bart흢omiej Potaczek. |
| 발행사항 | Birmingham, UK : Packt Publishing, 2019. |
| 형태사항 | 1 online resource : illustrations |
| 소장본 주기 | Added to collection customer.56279.3 |
| ISBN | 1838648836 9781838648831 |
| 내용주기 | Cover; Title Page; Copyright and Credits; About Packt; Contributors; Table of Contents; Preface; Chapter 1: Pyspark and Setting up Your Development Environment; An overview of PySpark; Spark SQL; Setting up Spark on Windows and PySpark; Core concepts in Spark and PySpark; SparkContext; Spark shell; SparkConf; Summary; Chapter 2: Getting Your Big Data into the Spark Environment Using RDDs; Loading data on to Spark RDDs; The UCI machine learning repository; Getting the data from the repository to Spark; Getting data into Spark; Parallelization with Spark RDDs; What is parallelization? Basics of RDD operationSummary; Chapter 3: Big Data Cleaning and Wrangling with Spark Notebooks; Using Spark Notebooks for quick iteration of ideas; Sampling/filtering RDDs to pick out relevant data points; Splitting datasets and creating some new combinations; Summary; Chapter 4: Aggregating and Summarizing Data into Useful Reports; Calculating averages with map and reduce; Faster average computations with aggregate; Pivot tabling with key-value paired data points; Summary; Chapter 5: Powerful Exploratory Data Analysis with MLlib; Computing summary statistics with MLlib Using Pearson and Spearman correlations to discover correlationsThe Pearson correlation; The Spearman correlation; Computing Pearson and Spearman correlations; Testing our hypotheses on large datasets; Summary; Chapter 6: Putting Structure on Your Big Data with SparkSQL; Manipulating DataFrames with Spark SQL schemas; Using Spark DSL to build queries; Summary; Chapter 7: Transformations and Actions; Using Spark transformations to defer computations to a later time; Avoiding transformations; Using the reduce and reduceByKey methods to calculate the results Performing actions that trigger computationsReusing the same rdd for different actions; Summary; Chapter 8: Immutable Design; Delving into the Spark RDD's parent/child chain; Extending an RDD; Chaining a new RDD with the parent; Testing our custom RDD; Using RDD in an immutable way; Using DataFrame operations to transform; Immutability in the highly concurrent environment; Using the Dataset API in an immutable way; Summary; Chapter 9: Avoiding Shuffle and Reducing Operational Expenses; Detecting a shuffle in a process; Testing operations that cause a shuffle in Apache Spark Changing the design of jobs with wide dependenciesUsing keyBy() operations to reduce shuffle; Using a custom partitioner to reduce shuffle; Summary; Chapter 10: Saving Data in the Correct Format; Saving data in plain text format; Leveraging JSON as a data format; Tabular formats -- CSV; Using Avro with Spark; Columnar formats -- Parquet; Summary; Chapter 11: Working with the Spark Key/Value API; Available actions on key/value pairs; Using aggregateByKey instead of groupBy(); Actions on key/value pairs; Available partitioners on key/value data; Implementing a custom partitioner; Summary |
| 요약 | In this book, you'll learn to implement some practical and proven techniques to improve aspects of programming and administration in Apache Spark. Techniques are demonstrated using practical examples and best practices. You will also learn how to use Spark and its Python API to create performant analytics with large-scale data. |
| 일반주제명 | SPARK (Computer program language) Application software -- Development. Big data. Electronic data processing. Python (Computer program language) Application software -- Development. Big data. Electronic data processing. Python (Computer program language) SPARK (Computer program language) |
| 언어 | 영어 |
| 기타형태 저록 | Print version:Lai, Rudy.Hands-On Big Data Analytics with Pyspark : Analyze Large Datasets and Discover Techniques for Testing, Immunizing, and Parallelizing Spark Jobs.Birmingham : Packt Publishing Ltd, 짤20199781838644130 |
| 대출바로가기 | http://search.ebscohost.com/login.aspx?direct=true&scope=site&db=nlebk&db=nlabk&AN=2094759 |
소장정보
- 소장정보
인쇄
| No. | 등록번호 | 청구기호 | 소장처 | 도서상태 | 반납예정일 | 예약 | 서비스 | 매체정보 |
|---|---|---|---|---|---|---|---|---|
| 1 | WE00017062 | 004.2 | 가야대학교/전자책서버(컴퓨터서버)/ | 대출가능 |
|
카피라이터
