[WIP] Add basic support for PlayerDisplayMetrics.devicePixelRatio on macOS

[JustinFincher]

This is a working proof of concept of devicePixelRatio support on macOS standalone player. As that being said, the current progress may not suitable for direct production usage, but to provide an idea on how to do this on macOS.

Added Features

• Get devicePixelRatio on macOS via OSXDeviceScaleFactor method
• Subscribe to screen change event and notify Unity via ViewportMatricsChanged event

Changes

• Added a bundle library Runtime/Plugins/platform/osx/NSScreenUtils.bundle with its source project. The source project is named with a dot prefix to mark it as 'hidden' to Unity to avoid symbol colliding when building the standalone player.
• The corresponding interface addition on C# side, mainly in the UIWidgetsMessageManager.cs and DisplayMetrics.cs.

How this works

• Get devicePixelRatio
  This is achieved by providing OSXDeviceScaleFactor the NSScreenUtils.m file:

float OSXDeviceScaleFactor()
{
    NSArray *ar = [NSApp orderedWindows]; // Get all NSWindow of the Unity player app.
    NSWindow *window = [ar objectAtIndex:0]; // Get the first window. Usually, there would be only one NSWindow in the app startup process unless there is another native plugin doing NSWindow related work. 
    NSScreen *screen = window.screen; // Get the NSScreen where the windows is at
    if (!screen) {
        screen = [NSScreen mainScreen]; // If the screen reference is NULL, get main screen instead.
    }
    if (screen)
    {
        return screen.backingScaleFactor; // Access the backingScaleFactor, this is available on macOS 10.7+. 
    }
    return 1;
}

• Subscribe to screen change event

This is a bit tricky because:

1. Standalone builds don't provide an export project option, so we cannot use the post-build attribute to modify the exported Xcode project. Instead, we build a native plugin to modify the behavior of NSWindow with the dynamic language feature of Objective-C (also called Method Swizzling).

#import "NSWindow+NSScreenUtils.h"
#import <objc/runtime.h>
#import "UIWidgetsMessageManager.h"

@implementation NSWindow (NSScreenUtils)


// Using method swizzling to inject notification on library load. Notice this bundle should be marked as 'Load on Start' in the Unity inspector panel
+ (void)load {
    NSLog(@"NSWindow + NSScreenUtils Load");
    static dispatch_once_t onceToken;
    dispatch_once(&onceToken, ^
                  {
                      NSString *bundleIdentifier = [[NSBundle mainBundle] bundleIdentifier];
                      if ([bundleIdentifier isEqualToString:@"com.unity3d.UnityEditor5.x"]) {
// if bundle id is Unity, then don't exchange method imp.
                      }
                      else
                      {
// Exchange method implementation to override the init method of NSWindow
                          [UIWidgetsMessageManager getInstance];
                          NSLog(@"NSWindow + NSScreenUtils Enabled for %@",bundleIdentifier);
                          [self exchangeClassMethodMethod:@selector(initWithContentRect:styleMask:backing:defer:) with:@selector(utils_initWithContentRect:styleMask:backing:defer:)];
                          [self exchangeClassMethodMethod:@selector(initWithContentRect:styleMask:backing:defer:screen:) with:@selector(utils_initWithContentRect:styleMask:backing:defer:screen:)];
                      }
                  });
}

+ (void)exchangeClassMethodMethod:(SEL)originalSelector with:(SEL)swizzledSelector
{
    // some magic stuff
}

#pragma mark - Method Swizzling

- (instancetype)utils_initWithContentRect:(NSRect)contentRect styleMask:(NSWindowStyleMask)style backing:(NSBackingStoreType)backingStoreType defer:(BOOL)flag screen:(NSScreen *)screen
{
    NSWindow * instance = [self utils_initWithContentRect:contentRect styleMask:style backing:backingStoreType defer:flag screen:screen];
    if (instance) {
        NSLog(@"utils_initWithContentRect:styleMask:backing:defer:screen:");
// Here we make NSWindow subscribe to a notification when initializing
        [[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(onWindowDidChangeBackingProperties:) name:NSWindowDidChangeBackingPropertiesNotification object:nil];
    }
    return instance;
}
- (instancetype)utils_initWithContentRect:(NSRect)contentRect styleMask:(NSWindowStyleMask)style backing:(NSBackingStoreType)backingStoreType defer:(BOOL)flag
{
    NSWindow * instance = [self utils_initWithContentRect:contentRect styleMask:style backing:backingStoreType defer:flag];
// Here we make NSWindow subscribe to a notification when initializing (again, as there are 3 initializer in NSWindow)
    if (instance) {
        NSLog(@"utils_initWithContentRect:styleMask:backing:defer:");
        [[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(onWindowDidChangeBackingProperties:) name:NSWindowDidChangeBackingPropertiesNotification object:nil];
    }
    return instance;
}

#pragma mark - Callback
// Send to Unity that ViewportMatricsChanged (Matrics should be Metrics, since ViewportMatricsChanged is used in the current java implementation we would just follow the name)
- (void)onWindowDidChangeBackingProperties:(NSNotification *)notification
{
    NSLog(@"onWindowDidChangeBackingProperties");
    [[UIWidgetsMessageManager getInstance] UIWidgetsMethodMessage:@"ViewportMatricsChanged" :@"UIWidgetViewController.keyboardChanged" :[NSArray array]];
}
@end

2. There seems to lack a UnitySendMessage interface in macOS player, at least not accessible like Android (import unity-classes.jar and call 'com.unity3d.player.UnityPlayer.UnitySendMessage') or iOS (import UnityInterface.h). So I have to build a custom UnitySendMessage clone and name it UnityOSXSendMessage. If there is an official way to call UnitySendMessage in macOS plugins then please ignore this issue.

// minimal UnitySendMessage clone
static UnityOSXCallback callback = NULL; 
void LinkUnityOSXCallback(UnityOSXCallback externalCallback)
{
    callback = externalCallback;
}
void UnityOSXSendMessage(const char *name,const char *method,const char *arg)
{
    if (callback) {
        callback(name,method,arg);
    }
}

void Awake() {
#if UNITY_STANDALONE_OSX
            // Call native code to construct a UnitySendMessage clone for macOS player. If there is an official way to call UnitySendMessage in macOS plugin please replace this.  
            LinkUnityOSXCallback((namePtr, methodPtr, argPtr) => {
                string name = Marshal.PtrToStringAuto(namePtr);
                string method = Marshal.PtrToStringAuto(methodPtr);
                string arg = Marshal.PtrToStringAuto(argPtr);
// Find a GameObject that have the name set previously by UIWidgetsMessageSetObjectName
                GameObject foundGameObject = GameObject.Find(name);
                if (foundGameObject != null) {
                    foundGameObject.SendMessage(method,arg);
                }
            });
#endif
        }

As the UnityOSXSendMessage link is established, we now send a "ViewportMatricsChanged" message whenever the screen change notification is triggered. The "UIWidgetViewController.keyboardChanged" argument is copied from the Java implementation and may need to change to a more suitable name.
In the Update function of DisplayMetrics.cs I added a line to invalidate the current scale factor, then in the next cycle UIWidgets would call OSXDeviceScaleFactor to get a new scale value.

Concern & To-do

• Though the ViewportMatricsChanged event is only called when the native screen change notification is triggered, the OSXDeviceScaleFactor function would be called every frame because displayMetrics.Update is called in UIWidgetsPanel.Update(), which is expensive on CPU usage. I wonder if there is a solution to call displayMetrics.Update only if needed.
• Some failsafe check is needed to handle corner cases like 2 NSWindows on launch.
• Is there an official UnitySendMessage on the standalone player? Use the official UnitySendMessage if possible would require less code in my implementation.

Reference

• Apple Doc - NSScreen.backingscalefactor
• Method Swizzling
• Apple Doc - Handle Dynamic Changes in Window Resolution Only When You Must

Demo Video

demo.mp4.zip

[zhuxingwei]

hi, sorry for this late response. Thanks your efforts and this PR looks great!

As to your concerns, (1) yes, we should not write the dirty mark in every frame update. I have made some changes in #243 to fix this; (2) do you have any idea on this issue? maybe we can come up with some fallback solutions? (3) since we have implemented UIWidgetsMessageManager for ios and android already, it is ok to use our own UIWidgetsMessageManager on Mac too.

[JustinFincher]

@zhuxingwei 1) Alright, I will be looking into #243 to get this PR fully working and qualified to be merged when I am available. Also currently I am in a TOFEL school so probably cannot do much about this issue right now, sorry! 2) 2x NSWindow situation is actually rare (if you don't count the standalone resolution dialog). If so, there would be extra work but still can be solved on the native side. 3) Great, though I would like my implementation of UIWidgetsMessageManager to be reviewed by you guys as my implementation is just a POC.