Detect screenshare rotation

[davidliu]

No description provided.

[changeset-bot]

🦋 Changeset detected

Latest commit: ccd01e3

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
client-sdk-android	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[bennnjamin]

These changes look very similar to what is required to fix this issue #535. Especially since it's possible for the device to be in landscape while screensharing when switching between apps. I would greatly appreciate if you took another look at a robust orientation handling solution for the camera in addition to screensharing.

[davidliu]

@bennnjamin As I noted in #535, the camera is already following the rotation of the device's UI, so I'm not sure what's actionable there.

[bennnjamin]

@davidliu it doesn't follow the device's UI when switching apps or when there is no foreground activity. It's certainly jarring to a subscriber to see the camera swapping between landscape and portrait orientation during normal operation of the phone. The expectation from a UX perspective is that the video orientation matches the device even when switching apps or multitasking.

The action is to use OrientationListener instead of WindowManager for camera video track orientation like in this pull request but even better would be to allow the developer override the frame orientation. Then it's up to them if they would like to override the default behavior with a custom OrientationListener.

[davidliu]

even better would be to allow the developer override the frame orientation.

@bennnjamin that's possible to do with a VideoProcessor, though you shouldn't have to for rotation. If you can provide a repro example, I'll gladly recheck it and provide a fix if needed, but it was working fine for me.

[bennnjamin]

@davidliu I appreciate your willingness to help with this. Perhaps I was looking in the wrong place to override rotation. Can you link to a method that is overridable? Here is are the methods I found for frame rotation but they are all private.

client-sdk-android/livekit-android-camerax/src/main/java/livekit/org/webrtc/CameraXSession.kt

Line 305 in dcbb36b

private fun getFrameOrientation(): Int {

And in WebRTC:

https://github.com/webrtc-sdk/webrtc/blob/b99fd2c270361aea2d458e61ac4a4cd2443bdbf6/sdk/android/src/java/org/webrtc/Camera2Session.java#L413

https://github.com/webrtc-sdk/webrtc/blob/b99fd2c270361aea2d458e61ac4a4cd2443bdbf6/sdk/android/src/java/org/webrtc/CameraSession.java#L42

[bennnjamin]

@davidliu kindly checking to see if you are able to provide further guidance on the correct place to override frame rotation for Camera2 and Camerax sessions