Troubleshooting Flutter <-> Unity integration

Recently, my colleague Saraswati Dwivedi and I were asked to help solve a problem on a project that is building an app in Flutter.

The problem

The client was trying to embed a Unity game into their Flutter app, but were experiencing problems on Android — poor performance, tap & swipe events not being recognized, etc. The project was using the Flutter Unity Widget to launch the Unity game from within Flutter.

Our approach

We did not have access to the Flutter app itself, however, we had been sent a native Android demo application that was auto-generated by Unity. We tested this to establish a baseline — the game appeared to work fine on all of our devices — a Google Pixel 3 and a Samsung S22 Ultra.

We had a theory, but we would need to test it out — we wondered if we launched the Unity game from within the native Android code (instead of from Flutter directly using the widget) if that would resolve the problems. However, we would need to test this assumption and prove whether it would work.

Our next step was actually two that we took in parallel. Saraswati Dwivedi worked on building a sample Flutter application and displaying a native Android screen — using the Platform View functionality.

I worked on taking the Unity game and trying to create our own sample native Android application — neither of us had used Unity before, so it was important for us to learn how to integrate Unity into a native app so we could get things wired up and verify whether it performed properly.

We had been given an exported Android library version of the Unity game, however, I ran into a LOT of problems trying to get this to build — mismatched Android/Gradle/NDK versions, and some ever-so-fun obscure low-level compiler errors.

One of my helpful colleagues, Earl, advised me (while on vacation!) to export the Android library from Unity source myself, which would hopefully work more effectively. Thankfully it did, although I learned some rather surprising things when trying to put the project into a repo so I could send the project to others (more on that later).

So, I needed to get the Unity editor up and running — it took a few attempts — it’s important to install Unity Hub first, because if you directly install Unity Studio it prompts you for license information, and Unity Hub serves as a Unity version manager, as each project may use a slightly different version… But, once we got the tools installed, we were able to successfully open the Unity project for the game.

One of the most helpful things I did next was reading through Unity’s official guide for integrating a Unity game into a native Android app. This showed me the structure of what was exported from Unity, and what steps I needed to take to integrate the game with the app (I highly recommend following this as there are some finicky configuration steps you need to take to get it to work). I actually started by following their tutorial step-by-step (with their own sample Unity game) first just to help build my knowledge, and was able to get it working!

Next, I exported the actual Unity game and tried following the same steps to run it — but ran into an unfortunate problem — the game would crash as soon as I attempted to launch it! After a very helpful reminder from another colleague (you know, actually read the crash log and try to interpret what was happening), I was able to do some research and stumble across this open issue filed against Unity’s demo project — you need to have at least one string defined in Android’s strings.xml resource file, otherwise, the game will crash!

Once I added that simple string, the moment of truth finally arrived, and I was able to launch the game successfully!!

Next up though, we needed to see if we could call this from Flutter.

This is where Sara’s work proved essential — she had built out a simple native Android view using Flutter’s Platform Views, and, thankfully, launching Unity on Android is as simple as just starting an Intent for a particular Activity. We worked together to modify her Flutter project to include the Unity game as a native Android module (following all of the steps I had just proven out in a pure native Android app), and then we launched the Activity from the native Android view, and it worked!!! Best of all, this configuration had none of the problems of the original version using the Flutter Unity Widget (poor performance and broken user input events).

Refining the solution

However, this was only an intermediate step on our journey — this solution still involved the user clicking a button in Flutter, then being shown a native Android screen, and clicking a second button. We wanted to see if it was possible to remove the intermediate screen.

The way we ended up solving this was to use Flutter’s MethodChannel API to call code within the native Android part of the application (without necessarily displaying UI) — from there, we were able to get a reference to the current Activity/Context, and form our Intent to launch the Unity game! This also worked like a charm, and we were thrilled!

I did try one thing that did not work — using the Android Intent Plus library to try launching the Intent directly from Flutter. Unfortunately, no matter how many different combinations I tried, I could not properly form the Intent — I always got some form of a “Activity not found” exception. I’m not sure if this was a malformed Intent problem, the Manifest not configured properly, or even if it had something to do with the native Android Unity Activity not being loaded yet (almost like a lazy-loading issue).

Applying this to iOS

The client wanted us to take the same approach for iOS and Android, so Saraswati Dwivedi worked on implementing this same general strategy on iOS (she also wrote this portion of the article!)

In a Flutter app, launching a Unity game from within the native iOS code is as complicated as it sounds. Our approach to solve this complicated problem was to break it into smaller parts and solve them step by step:

1. Export a standalone iOS app from Unity project and run it successfully on a device.
2. Embed the Unity project in a native iOS app and launch Unity game on button click.
3. Creating a bridge between Flutter and native iOS code using Platform View and using it to present a simple native iOS view on a button click in the Flutter app.
4. Finally, removing the simple native iOS view and launching the Unity game instead.

1. Export a standalone iOS app from Unity project and run it successfully on a device.

In theory this should be the simplest of all the tasks, however it took the largest amount of my time. I was trying to export the standalone iOS app from the Unity project but every attempt to run the game in a simulator failed. I tried to make changes in Unity’s player settings and run it with different settings but every attempt failed. Even when the app built successfully it crashed while launching the game. A helpful colleague suggested to run the game on a physical device instead of a simulator, and I tried doing that but still had no success in launching the game. After some more attempts and investigation I realized that I was experiencing some kind of cache issue. I followed these steps to resolve it:

• Clear cache in Unity and derived data from XCode
• Delete extra projects from everywhere and Unity
• Restarted iPhone and Mac
• Started project in Unity making sure the latest Unity version was selected. (For me it was Unity editor version 2023.1.6f1).
• Exported iOS project for device without changing any player settings.

And finally I was able to run the game on a device!

2. Embed the Unity project in a native iOS app and launch Unity game on button click.

I tried to follow these instructions to embed the Unity project in a native iOS app, but was getting build issues. Then Matthew Groves suggested I follow the official Unity instructions for iOS, just like he did for Android, which worked like a charm and I was quickly able to integrate Unity as a library in a native iOS app. I then used this simple example to launch the Unity game on button click.

3. Creating a bridge between Flutter and native iOS code using Platform View and using it to present a simple native iOS view on a button click in the Flutter app.

Flutter provides detailed guidance on how to use Platform View to present native iOS View, using which I was successfully able to present a simple native iOS view on a button click in the Flutter app.

4. Finally, removing the simple native iOS view and launching the Unity game instead.

This was the last piece of the puzzle and to solve this I had to rework the Unity initialization and loading and overall approach I took in step 2 as a part of solution refinement. While following this example, if I call initAndShowUnity() on button click itself, it was blocking the main thread for a few seconds resulting in a minor lag during the game launching transition. Calling initAndShowUnity() from application didFinishLaunchingWithOptions seems to solve the thread issue but was not the accepted solution since this loaded the game when the app launched (along with the sound effect). I finally referred to this example to create a utility class and the flutter_unity_widget native code to see the approach to follow, and I was finally able to embed Unity in the native iOS side and launch it from the Flutter app using the bridge we created between Flutter and native iOS code using Platform View.

Other Learnings

1. As I had mentioned previously, we did not have access to the client’s source code, so someone actually working on the project had to integrate our changes into the official app. We ran into a really interesting snag — they got everything configured, but we had the exact same performance/functionality issues. What we found is that there is a bug in Unity for Android where using a non-default compression algorithm (LZ4 or LZ4HC) causes problems. Unity has a TON of different settings, so we were lucky that we found this fairly quickly by comparing how I had exported the Unity game vs. what this other person had done. Here’s another report from Unity’s forums about the same issue.
2. When I was getting ready to put all of my work on Android into a repo so the person working on the project could easily access my work, I ran into a very unexpected problem. I was trying to move files around and clean things up when I discovered that the exported Android Unity library contains absolute file paths (/Users/myusername/project/…). I am quite surprised this works this way — I haven’t yet found if there’s a project setting to override this, or if people always export the Unity library as part of their CI pipeline, or have a script to convert the file paths to relative ones, but, beware of this!

Thanks for reading and please follow our Digital Products Tech Tales publication to learn more about what our team is working on!