Ep.31 — Helping the hearing impaired using Apple Watch’s ML capabilities

  1. Sound classification is surprisingly mature with a low rate of false positives. In my use case, audio stream from the mic is analyzed on the fly to look for safety related sounds such as car horns and sirens. The detection time is pretty good too, less than 10ms.
  2. Haptic feedback is really useful, but it is often forgotten during user experience design. Engineers have fine-grained control over the type of vibrations on iOS using the UIFeedbackGenerator framework (it’s not available on watchOS yet.) BuzzWatch can only use the standard vibration for all types of important sounds. I couldn’t get it to work like Apple Maps navigation where you can customize the haptic experience. I hope Apple would improve watchOS in this area. Haptic feedback is very handy for wearables. (I wonder if Wear OS 3.5 on Android is better… time for a Pixel Watch?)
  3. One important reminder on using Haptics with AVAudioSession: When you are recording audio, system sounds / haptics (e.g. Notifications) are disabled. You can override it using session.setAllowHapticsAndSystemSoundsDuringRecording(true). This one was tricky to discover because audio recording isn’t really related to haptics stuff and it’s hard to make that linkage when debugging.
  4. SwiftUI is being forced upon us for more and more use cases. It started with Widget development last year and now with watchOS 9.0, you can’t even start a new watchOS project using Storyboard anymore on Xcode 14. If you have existing code written to WatchKit and the traditional storyboard-based UI, it’s time to start planning a migration. In this project, I started out with Xcode 13 with storyboard; but eventually had to give up and redo everything on the watch app using SwiftUI.
  5. Don’t assume the OS always behave correctly, especially with .0 releases. I made this mistake when trying to get custom UserNotification working on watchOS. After much head-banging-against-the-wall, it turned out that there was nothing wrong with the code. Custom notification view works consistently on watchOS 8.5, but watchOS 9.0 just doesn’t behave. I ended up filing a bug report to Apple, hoping they would address it at some point. ¯\_(ツ)_/¯
BuzzWatch (https://apps.apple.com/us/app/buzzwatch/id6443629077)

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store