Here are some things that were mentioned during the Platforms State of the Union at WWDC 2019.
macOS Catalina
- DriverKit - kernel extensions but in userspace
Explicit User Authorizations for:
- Keystroke recording
- Screen capture
- Screen recording
- User Data
- Document
- Desktop
- Downloads
- iCloud Drive
- Removeable media
- Network volumes
watchOS 6
- Independence from the iPhone
- Audio Streaming
- Extended Runtime
- SwiftUI support
Privacy
Sign In with Apple
- Real User Indicator based on account history (determine if another verification step should be performed)
- Works on the Web and across all Apple platforms
ML
- image saliency (user interest heat maps for auto-cropping)
- text recognition (printed text transcription)
- word embeddings (get synonyms of words for search purposes)
- speech api on-device (recognize speech)
- speech saliency (get the cadence and pitch of spoken words)
CoreML
- Personalized, background-updated machine learning models personalized on each user's device.
CreateML
- macOS app for building models without code with templates, and testing them with various inputs from built-in sensors on iOS and Mac.
Siri Shortcuts
- Shortcut phrase suggestions (Add to Siri)
- Conversational (parametrized) shortcuts
- Shipped with iOS and iPadOS
- Automation (smart shortcut triggers)
- Multi-step, multi-app shortcuts
ARKit
- Apple Pay integration (instant buy)
- Simultaneous front and back camera usage
- Motion capture
- People occlusion
RealityComposer
- WYSIWYG interface for laying out 3D AR experiences for Apple devices
- An iOS and Mac application
RealityKit
- An engine for 3D AR experiences
- Multi-threaded renderer
- Synchronizes with camera data for realistic results
Metal
- Supported fully by the iOS Simulator on Mac
- Metal Memory Debugger to understand resource allocation
- Metal Indirect Compute Encoding (offloading more tasks from the CPU to the GPU)
- Better raytracing support
- Peer Group API (GPUs can share data with each other without clogging the slower PCI system bus)
Thanks for reading!
Discussion
As a user, I think Sign In with Apple is going to be great for privacy.
For sure! It'll also help complete the loop for applications that can't store data with iCloud but still want to be at home on Apple's platforms.