<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Max Cobb on Medium]]></title>
        <description><![CDATA[Stories by Max Cobb on Medium]]></description>
        <link>https://medium.com/@maxxfrazer?source=rss-2f5df90bc0eb------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*L6TXP8CxR9RJNFa93oWGVw.jpeg</url>
            <title>Stories by Max Cobb on Medium</title>
            <link>https://medium.com/@maxxfrazer?source=rss-2f5df90bc0eb------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 14:40:47 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@maxxfrazer/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com" rel="hub"/>
        <item>
            <title><![CDATA[SwiftUI Video Streaming]]></title>
            <link>https://maxxfrazer.medium.com/swiftui-video-streaming-20c7abfd49f?source=rss-2f5df90bc0eb------2</link>
            <guid isPermaLink="false">https://medium.com/p/20c7abfd49f</guid>
            <category><![CDATA[live-streaming]]></category>
            <category><![CDATA[swiftui]]></category>
            <category><![CDATA[swift-programming]]></category>
            <category><![CDATA[agora]]></category>
            <category><![CDATA[streaming-video]]></category>
            <dc:creator><![CDATA[Max Cobb]]></dc:creator>
            <pubDate>Mon, 10 Jul 2023 10:31:56 GMT</pubDate>
            <atom:updated>2023-07-11T17:14:07.488Z</atom:updated>
            <content:encoded><![CDATA[<h4>Empowering SwiftUI with Seamless Live Streaming Capabilities</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZPC2aVvUGFrOOqT1OP4ZzQ.jpeg" /></figure><h3>Overview</h3><p>SwiftUI has emerged as a game-changer in app development, providing a modern and intuitive way to build user interfaces for various Apple platforms. While SwiftUI offers tremendous capabilities, there is a noticeable absence of robust solutions for live streaming applications. However, the SwiftUIRtc package fills this gap by offering a seamless integration of live streaming capabilities in SwiftUI. In this article, we explore how SwiftUIRtc empowers developers to create compelling live streaming applications using the power of SwiftUI.</p><h3>The Challenge of Live Streaming in SwiftUI</h3><p>Live streaming poses unique challenges in terms of real-time video rendering, media synchronization, and interactive features. SwiftUI, although a powerful UI framework, lacks native support for live streaming components. Developers often face the daunting task of integrating third-party libraries and complex workarounds using UIViewRespresentable over and over again to achieve live streaming functionality in their SwiftUI apps.</p><h3>A SwiftUI Live Streaming Solution</h3><p>The package <a href="https://agoraio-community.github.io/SwiftUIRtc/documentation/swiftuirtc">SwiftUIRtc</a> is designed to address awkward workarounds needed for live streaming in SwiftUI. It provides a small set of tools and SwiftUI components that enable developers to effortlessly incorporate live streaming capabilities into their SwiftUI applications. By leveraging <a href="https://agoraio-community.github.io/SwiftUIRtc/documentation/swiftuirtc">SwiftUIRtc</a>, developers can overcome the hurdles associated with live streaming integration and focus on delivering an exceptional streaming experience to their users.</p><p>The package uses the Agora Video RTC engine under the hood to ensure that the best network is used for this solution.</p><h3>Key Classes of SwiftUIRtc</h3><p>Although the SwiftUIRtc package is relatively light by design, its limited classes and methods are everything developers need to start working with live streaming applications right away.</p><h3>AgoraManager</h3><p><a href="https://agoraio-community.github.io/SwiftUIRtc/documentation/swiftuirtc/agoramanager">AgoraManager</a> has several very important responsibilities, including session management and connection handling. <a href="https://agoraio-community.github.io/SwiftUIRtc/documentation/swiftuirtc/agoramanager">AgoraManager</a> will keep track of the users who join and leave the channel so that you as the developer know what IDs you need to render the video for. These IDs are tracked using delegate methods from the Agora RTC engine which stores them in a published variable, <a href="https://agoraio-community.github.io/SwiftUIRtc/documentation/swiftuirtc/agoramanager/allusers">allUsers</a>.</p><p>Join and leave channel methods:</p><ul><li><a href="https://agoraio-community.github.io/SwiftUIRtc/documentation/swiftuirtc/agoramanager/joinchannel(_:token:uid:info:)">joinChannel(_:token:uid:info:)</a></li><li><a href="https://agoraio-community.github.io/SwiftUIRtc/documentation/swiftuirtc/agoramanager/leavechannel(leavechannelblock:destroyengine:)">leaveChannel(leaveChannelBlock:destroyEngine:)</a></li></ul><p>Implemented delegate methods:</p><ul><li><a href="https://agoraio-community.github.io/SwiftUIRtc/documentation/swiftuirtc/agoramanager/rtcengine(_:didjoinedofuid:elapsed:)">rtcEngine(_:didJoinedOfUid:elapsed:)</a></li><li><a href="https://agoraio-community.github.io/SwiftUIRtc/documentation/swiftuirtc/agoramanager/rtcengine(_:didofflineofuid:reason:)">rtcEngine(_:didOfflineOfUid:reason:)</a></li></ul><h3>AgoraVideoCanvasView</h3><p><a href="https://agoraio-community.github.io/SwiftUIRtc/documentation/swiftuirtc/agoravideocanvasview">AgoraVideoCanvasView</a> is a UIViewRepresentable struct that provides a view for displaying remote or local video in an Agora RTC session. With access to the engine through <a href="https://agoraio-community.github.io/SwiftUIRtc/documentation/swiftuirtc/agoramanager">AgoraManager</a>, <a href="https://agoraio-community.github.io/SwiftUIRtc/documentation/swiftuirtc/agoravideocanvasview">AgoraVideoCanvasView</a> displays local and remote video streams when provided with just a desired ID.</p><h3>Example</h3><p>Using just the two classes described above, we can easily add an AgoraManager that keeps track of all the users in the scene and an AgoraVideoCanvasView for each user listed in the allUsers property as follows:</p><pre>import SwiftUIRtc<br>import AgoraRtcKit<br><br>struct ScrollingVideoCallView: View {<br>    @ObservedObject var agoraManager = AgoraManager(appId: &lt;#AppId#&gt;, role: .broadcaster)<br>    var channelId: String = &quot;test&quot;<br>    var body: some View {<br>        ScrollView {<br>            VStack {<br>                ForEach(Array(agoraManager.allUsers), id: \.self) { uid in<br>                    AgoraVideoCanvasView(manager: agoraManager, uid: uid)<br>                        .aspectRatio(contentMode: .fit).cornerRadius(10)<br>                }<br>            }.padding(20)<br>        }.onAppear {<br>            agoraManager.joinChannel(channelId, token: &lt;#Agora Temp Token#&gt;)<br>        }.onDisappear {<br>            agoraManager.leaveChannel()<br>        }<br>    }<br>}</pre><p>When displayed in a ScrollView as above, the different video feeds will automatically appear in a scrolling list view, scaling to however many users join the channel.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/256/1*Gpnrj8c7o51G8AV8E8q_Ww.gif" /></figure><h3>Extending SwiftUIRtc</h3><p>All of the methods in AgoraManager and AgoraVideoCanvasView are marked as open, and the library is open source. That means that each of these classes can be extended with subclasses to add functionality tailored to the exact needs of any SwiftUI application.</p><p>For example, if you need to record the quality of incoming streams, you can subclass AgoraManager as follows:</p><pre>public class CallQualityManager: AgoraManager {<br>    /// A dictionary mapping user IDs to call quality statistics.<br>    @Published public var callQualities: [UInt: String] = [:]<br>}</pre><p>And add a couple of additional delegate methods:</p><pre>extension CallQualityManager {<br>    public func rtcEngine(_ engine: AgoraRtcEngineKit, remoteVideoStats stats: AgoraRtcRemoteVideoStats) {<br>        self.callQualities[stats.uid] = &quot;&quot;&quot;<br>        Received Bitrate = \(stats.receivedBitrate)<br>        Frame = \(stats.width)x\(stats.height), \(stats.receivedFrameRate)fps<br>        Frame Loss Rate = \(stats.frameLossRate)<br>        Packet Loss Rate = \(stats.packetLossRate)<br>        &quot;&quot;&quot;<br>    }<br><br>    public func rtcEngine(_ engine: AgoraRtcEngineKit, localVideoStats stats: AgoraRtcLocalVideoStats, sourceType: AgoraVideoSourceType) {<br>        self.callQualities[self.localUserId] = &quot;&quot;&quot;<br>        Captured Frame = \(stats.captureFrameWidth)x\(stats.captureFrameHeight), \(stats.captureFrameRate)fps<br>        Encoded Frame = \(stats.encodedFrameWidth)x\(stats.encodedFrameHeight), \(stats.encoderOutputFrameRate)fps<br>        Sent Data = \(stats.sentFrameRate)fps, bitrate: \(stats.sentBitrate)<br>        Packet Loss Rate = \(stats.txPacketLossRate)<br>        &quot;&quot;&quot;<br>    }<br>}</pre><p>To display those call quality data, a simple change to ScrollingVideoCallView (above) adds an overlay to each <a href="https://agoraio-community.github.io/SwiftUIRtc/documentation/swiftuirtc/agoravideocanvasview">AgoraVideoCanvasView</a>:</p><pre>AgoraVideoCanvasView(manager: agoraManager, uid: uid)<br>    .aspectRatio(contentMode: .fit).cornerRadius(10)<br>    .overlay(alignment: .topLeading) {<br>        Text(agoraManager.callQualities[uid] ?? &quot;no data&quot;).padding()<br>    }</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*cl252N3fYbs8mT_NvL_usw.jpeg" /></figure><h3>Conclusion</h3><p>SwiftUIRtc brings simplicity and ease of use to the process of integrating live streaming capabilities into SwiftUI applications. By leveraging AgoraManager and AgoraVideoCanvasView, you can add video streams through the Agora RTC SDK, without having to deal with non-SwiftUI classes.</p><p>Use SwiftUIRtc in your iOS applications today by adding the package with:</p><pre>https://github.com/AgoraIO-Community/SwiftUIRtc.git</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=20c7abfd49f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What’s New in Apple AR/VR 2023]]></title>
            <link>https://maxxfrazer.medium.com/whats-new-in-apple-ar-vr-2023-70fcafd8e574?source=rss-2f5df90bc0eb------2</link>
            <guid isPermaLink="false">https://medium.com/p/70fcafd8e574</guid>
            <category><![CDATA[realitykit]]></category>
            <category><![CDATA[augmented-reality]]></category>
            <category><![CDATA[wwdc-2023]]></category>
            <category><![CDATA[provision]]></category>
            <category><![CDATA[visionos]]></category>
            <dc:creator><![CDATA[Max Cobb]]></dc:creator>
            <pubDate>Wed, 07 Jun 2023 01:16:57 GMT</pubDate>
            <atom:updated>2023-06-07T04:36:46.442Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Apple Vision Pro, visionOS, SwiftUI improvements in RealityKit, and more highlights from Apple’s WWDC Developer Conference 2023.</blockquote><figure><img alt="Banner image showing apple vision pro headset and several WWDC23 stickers as 3d objects" src="https://cdn-images-1.medium.com/max/1024/1*PVii6xsoCBkpw-8HsqeN8g.png" /></figure><p>It’s been six years since ARKit’s debut at WWDC 2017, and we’ve seen big milestones since then; but none come close to what was announced at WWDC 2023 this week.</p><p>From a full AR/VR simulator, to Apple Vision Pro, to full skeletal hand tracking, you’ll find the AR/VR highlights from the main keynote listed below; as well as other things hiding around in Apple’s beta API References. It won’t go into too much technical details on how to implement some of these new features and new frameworks, you’ll have to <a href="https://twitter.com/maxxfrazer">follow me on twitter</a>.</p><p>Although many of the APIs are not yet public (expected later in June 2023), we’ll cover an outline of the main new features coming our way soon.</p><h3>ARKit</h3><p>Starting with the core of AR at Apple, there only seems to be one feature available for now, and only on the newly introduced OS.</p><h4>Skeletal Hand Tracking</h4><p>As well as tracking surfaces, images and bodies, ARKit will now have skeletal hand tracking, starting with visionOS. This seems to be enabled due to a new device that has multiple cameras to better percieve depth.</p><figure><img alt="A  hand with a globe floating above" src="https://cdn-images-1.medium.com/max/1024/1*pQWbsIcfHsqr541BxrN-3g.gif" /></figure><h3>RealityKit</h3><p>Not known to be the fastest horse in terms of rolling out features, especially when compared to its older sister <a href="https://developer.apple.com/scenekit/">SceneKit</a>. But this year there were some much needed advancements, which were of course necessary for a certain piece of hardware.</p><blockquote>It currently unclear whether these updates will be extended to macOS and iOS, rather than just visionOS.</blockquote><h4>RealityKit x SwiftUI</h4><p>With the introduction of a whole new SwiftUI view, RealityView, RealityKit can now fully take advantage of SwiftUI layouts, and vice versa.</p><p>You can also add SwiftUI views into RealityKit in the form of attachments.</p><figure><img alt="Rendering of an island dipslayed in AR, with a couple of SwiftUI views added as attachments" src="https://cdn-images-1.medium.com/max/910/1*eIibq-ncg5ub3QvrkNi1iw.gif" /></figure><p>This now allows us to add content with sharp, readable text, especially those with complex layouts and animations.</p><h4>MaterialX</h4><p>RealityKit is adopting the open standard MaterialX for surface and geometry shaders.</p><p>MaterialX is supported across a broad variety of tools. Meaning people already familiar with those tools can now write already shaders in RealityKit, as well as those already using RealityKit can now have more developer tools and countless shader examples at their disposal.</p><h3>visionOS</h3><p>Apple’s exciting new operating system, visionOS brings many of the AR/VR concepts that RealityKit developers may be familiar with together in a stunning new headset dubbed “Vision Pro”. While many of the</p><h4>Fundamentals</h4><p>There are a few key objects that you can create within Vision Pro’s OS, visionOS:</p><ul><li>Windows, planes in space.</li><li>Volumes, which can hold 3d objects, such as a game board.</li><li>Full Space, which are fully immersive environments, and can not necessarily be manipulated by the user.</li></ul><figure><img alt="Screenshot of the platforms state of the union presentation showing the three fundamentals of visionOS" src="https://cdn-images-1.medium.com/max/1024/1*7Po7tRsuRSrZFMSKVLNFdA.jpeg" /></figure><h4>Apps</h4><p>Existing iOS/iPadOS apps are supported as a scalable 2d window shown in 3d space on visionOS. This window keeps their original look and feel.</p><figure><img alt="Existing iPad application, brought into a 3d scene with visionOS" src="https://cdn-images-1.medium.com/max/1024/1*HljRrktlCxtInAWVc-wmAg.gif" /></figure><p>If you simply add the build target of visionOS to your application, it immediately gets those nice visionOS materials, a fully resizable window, and displays the eye focus of the user, as you find in all visionOS applications.</p><figure><img alt="Updated iPad application, with visionOS materials" src="https://cdn-images-1.medium.com/max/1024/1*v1j_j1L0LNnXu0mCM2wNJg.gif" /></figure><p>With visionOS SwiftUI or UIKit can be used display content, RealityKit for 3d content, and ARKit to understand the user’s space. The apps designed specifically for visionOS can create a collection of windows, volumes, and spaces.</p><h4>Simulator</h4><p>My favourite part of the developer tooling at WWDC 2023. This simulator lets you move and look around the scene, and interact with the app by simulating the system gestures. There are several different builtin scenes, with both day and night coditions to help better understand how users will interact with your apps.</p><figure><img alt="visionOS simulator, showing an application in the middle, camera is moving towards it and rotating slightly to show depth" src="https://cdn-images-1.medium.com/max/1024/1*4udH-K_FpucRwa_-XAn-3A.gif" /></figure><h4>Mac Virtual Display</h4><p>Simply by looking at your Mac through a Vision Pro, your Mac’s screen will pop out and instead a high fidelity 4k monitor will appear in its place. This enables a whole new end-to-end development experience, where you can code, test, and debug all in the same place.</p><figure><img alt="Visualisation of a mac window on the right containing Xcode, and the built visionOS application on the left. macbook with blank screen below both." src="https://cdn-images-1.medium.com/max/1024/1*-5WlcMp23okZ_t0lwmy2WQ.jpeg" /></figure><p>Bluetooth keyboards and mice can be used for data input across visionOS, so of course you won’t have to type on a virtual keyboard.</p><h4>Reality Composer Pro</h4><p>Using Reality Composer Pro, you can add and prepare 3d content for visionOS apps.</p><h3>What’s next?</h3><p>You may have noticed some sections are a little empty, I’ll be expanding upon the content over the next week or so as more videos and software are released.</p><p>To stay tuned, drop a follow on <a href="https://medium.com/u/2f5df90bc0eb">medium</a> as well as <a href="https://twitter.com/maxxfrazer">twitter</a>. I’m always posting about the latest advancements in AR/VR, and specifically those in the Swift ecosystem!</p><p>Drop some claps so I know you’re reading too 👏👏👏</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=70fcafd8e574" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Getting Started with RealityKit: Brilliant Colours]]></title>
            <link>https://maxxfrazer.medium.com/getting-started-with-realitykit-brilliant-colours-91a12936b773?source=rss-2f5df90bc0eb------2</link>
            <guid isPermaLink="false">https://medium.com/p/91a12936b773</guid>
            <category><![CDATA[wwdc]]></category>
            <category><![CDATA[ios]]></category>
            <category><![CDATA[realitykit]]></category>
            <category><![CDATA[swift]]></category>
            <category><![CDATA[augmented-reality]]></category>
            <dc:creator><![CDATA[Max Cobb]]></dc:creator>
            <pubDate>Mon, 15 May 2023 01:50:25 GMT</pubDate>
            <atom:updated>2023-05-18T09:05:46.258Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*iXUsnTfSERcyrhM7.png" /></figure><p>Have you ever noticed that the materials in RealityKit appear dull and lackluster? Even when using UnlitMaterials, the colors tend to look muted and unimpressive. <a href="https://medium.com/u/375ea8256d60">Andrew Hart</a> was asking me why this is the case, and more importantly, how to fix it for his <a href="https://twitter.com/andrewhartar/status/1647700548726996993?s=46&amp;t=q9hUleEJJkN6LvlXVyeeUQ">upcoming open source project</a>. In this article, I’ll explain my understanding, and a solution to achieve more vibrant colors in your RealityKit scenes.</p><h3>The Current Issue</h3><p>By default, the materials in RealityKit are designed to mimic real-world lighting conditions. This means that even unlit materials will have some degree of shading, resulting in colors that appear less saturated, and thus blend in more with the other objects.</p><p>While this approach may be appropriate for many use cases, such as architectural visualizations or product renderings, it can be frustrating when you’re trying to create a more stylized or vibrant scene, or when just want some objects to stand out amongst the other objects that should blend into the scene.</p><p>Take a look at this example, showing a simple sphere in a 3d scene. It looks okay, but in no way does that white colour stand out as a true white:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aAvE0lWw6mwCXq17diFjAw.png" /></figure><p>The actual pixel value of the cube shows it’s only 214 out of a maximum of 255 for true white.</p><p>The same can be said when applying a texture.<br> The colours are much more dull than they’re intended to be, which is frustrating when wanting to make objects in your AR scene that really pop.</p><h3>The Solution</h3><p>Fortunately, there’s a simple solution to this problem that doesn’t involve custom shaders, using the new(ish) material class <a href="https://developer.apple.com/documentation/realitykit/physicallybasedmaterial">PhysicallyBasedMaterial</a>.</p><p>This material type is actually made for simulating the appearance of real-world objects, but there’s a trick to making it appear just like you would expect an <a href="https://developer.apple.com/documentation/realitykit/unlitmaterial">UnlitMaterial</a> to act, which is to set its <a href="https://developer.apple.com/documentation/realitykit/physicallybasedmaterial/basecolor-swift.property">baseColor</a> and <a href="https://developer.apple.com/documentation/realitykit/physicallybasedmaterial/sheen-swift.property">sheen</a> to black, which then means the object will not respond to lighting, then apply your colour or texture to the <a href="https://developer.apple.com/documentation/realitykit/physicallybasedmaterial/emissivecolor-swift.property">emissiveColor</a>, and bump up the <a href="https://developer.apple.com/documentation/realitykit/physicallybasedmaterial/emissiveintensity">emissiveIntensity</a> to a value greater than its default of 1.</p><p>Here’s an example of what that might look like:</p><pre>var mat = PhysicallyBasedMaterial()<br>mat.baseColor = .init(tint: .black)<br>mat.sheen = .init(tint: .black)<br>mat.emissiveColor = .init(color: .white)<br>mat.emissiveIntensity = 2</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A3RG6oiSlaUpyoTZ2-goVw.png" /></figure><blockquote><em>The new material is on the right.</em></blockquote><p>Here’s what the white pixel values are in the scene:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IiiuwebN88laHA-HNOxPgw.gif" /></figure><p>Similarly, instead of setting the color in the emissiveColor, you can set a texture as such:</p><pre>mat.emissiveColor = .init(texture: MaterialParameters.Texture(<br>  try! .load(named: &quot;rgb_test&quot;)<br>))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pfUeC4aId8KOKUrmoaRWpA.png" /></figure><p>The above image shows an UnlitMaterial on the left, a SimpleMaterial in the middle, and a BoldMaterial on the right, using this image:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/259/1*VbZEX4kx8f95jMN2Mr3OLQ.png" /></figure><p>By making this small change, you’ll be able to achieve more vibrant and true-to-life colors in your RealityKit scenes, without sacrificing the benefits of an unlit material.</p><p>The type of BoldMaterial is included in <a href="https://github.com/maxxfrazer/RealityToolkit">RealityToolkit</a>, which is a lightweight set of tools to make RealityKit more powerful, with remote image and USDZ file loading, and now BoldMaterial too.</p><p>Here’s a few examples with other colours, with the unlit material on the left, simple material in the middle, then the adaptation of the physically based material on the right:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p4fCVTW25OVDndCQD5Nudw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PNnclTI7xWs6NubukNsSoA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Okoimwjr1ZSlPDDY32B5dQ.png" /></figure><h3>But What About Lighting?</h3><p>This technique works best for materials that have no response to lighting, giving them an appearance that really stands out. However, if you do what your materials to respond to lighting, a few small tweaks could help. These tweaks include keeping the sheen as .white, and increasing the baseColor to instead a dark gray.</p><p>Play around with <a href="https://developer.apple.com/documentation/realitykit/physicallybasedmaterial">PhysicallyBasedMaterial</a> to find the exact appearance you’re looking for in your RealityKit app.</p><h3>Conclusion</h3><p>While RealityKit is a powerful tool for creating immersive AR and VR experiences, things like this can sometimes make it more challenging than necessary to achieve the desired look and feel for your scenes. By using alternative material types, you can create more vibrant and engaging scenes that capture your user’s attention.<br> As mentioned above, check out the <a href="https://github.com/maxxfrazer/RealityToolkit">RealityToolkit</a> Swift Package to make adding bold materials to your project easy.</p><p>I hope that this simple guide has helped you achieve the vibrant colors you’re looking for in your RealityKit projects. Feel free to leave some claps below if you found this useful!</p><p>👏👏👏</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=91a12936b773" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying DocC with GitHub Actions]]></title>
            <link>https://maxxfrazer.medium.com/deploying-docc-with-github-actions-218c5ca6cad5?source=rss-2f5df90bc0eb------2</link>
            <guid isPermaLink="false">https://medium.com/p/218c5ca6cad5</guid>
            <category><![CDATA[github-actions]]></category>
            <category><![CDATA[swift]]></category>
            <category><![CDATA[documentation]]></category>
            <category><![CDATA[docc]]></category>
            <category><![CDATA[github]]></category>
            <dc:creator><![CDATA[Max Cobb]]></dc:creator>
            <pubDate>Wed, 19 Oct 2022 10:28:25 GMT</pubDate>
            <atom:updated>2023-06-02T13:35:42.806Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fk9tKOUZB2TlWXR6WShcyg.png" /></figure><h4>DocC with Swift Package Index</h4><p>To save you reading through this post when all you really want to do is get DocC for your Swift Package, check out <a href="https://blog.swiftpackageindex.com/posts/auto-generating-auto-hosting-and-auto-updating-docc-documentation/">this post from Swift Package Index</a>. The team at SPI literally build and host your documentation for your package if you add a short .spi.yml file to your project root.</p><h3><strong>What is DocC?</strong></h3><p>DocC is a documentation compiler for iOS applications or frameworks. It produces a rich API reference, as well as interactive tutorials, articles, and more.</p><p>Apple’s documentation does an excellent job of explaining DocC:</p><p><a href="https://developer.apple.com/documentation/docc">https://developer.apple.com/documentation/docc</a></p><p>Here’s an example of DocC builds from some repositories on GitHub:</p><ul><li><a href="https://messagekit.github.io/MessageKit/documentation/messagekit/">MessageKit</a></li><li><a href="https://maxxfrazer.github.io/RealityUI/documentation/realityui/">RealityUI</a></li><li><a href="https://agoraio-community.github.io/VideoUIKit-iOS/documentation/agorauikit/">Agora Video UI Kit</a></li></ul><p>DocC is not only the easiest way to create iOS docs; it’s best way to deliver your docs to iOS developers.</p><p>With the latest improvements from DocC we can now:</p><ul><li>Build docs for Objective-C products as well as Swift</li><li>Toggle between Objective-C and Swift</li><li>Filter within the documentation</li><li>Publish docs directly on GitHub Pages (static website)</li></ul><h3><strong>Build a DocC Archive</strong></h3><p>With a Swift Package open in Xcode, you can go straight to Product &gt; Build Documentation and Xcode will generate and show your documentation in a separate window. From this window you can export the .doccarchive file.</p><p>This can be used to host on your domain, or imported by someone else to show your package’s documentation.</p><p>Alternatively, the same can be achieved from the command line. Go to the root directory of your swift package, and run the following:</p><pre>xcodebuild docbuild -scheme MyPackage \<br>    -derivedDataPath docc \<br>    -destination &#39;generic/platform=iOS&#39;;</pre><blockquote>Swap out “MyPackage” with the name of your package or other scheme. For example, with the package <a href="https://github.com/maxxfrazer/RealityUI">maxxfrazer/RealityUI</a>, you would use RealityUI. Similarly, swap out the platform if you are building for a different platform.</blockquote><p><a href="https://medium.com/u/790f3c63913c">Dmitry</a> pointed out to me in the comments, you may also need to add these parameters to the xcodebuild script, in the case you’re build a doccarchive for a project, rather than a package:</p><pre>CODE_SIGN_IDENTITY=&quot;&quot; CODE_SIGNING_REQUIRED=NO</pre><p>As the derivedDataPath is set to &quot;docc&quot;, a new folder called docc will appear in your project root, which contains the doccarchive:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/486/1*iOPf36il2izjbNq1d2JyLQ.png" /></figure><h3>DocC Archive to Static Website</h3><p>If you want to host this on a static hosting site, there’s one more step you’ll have to make, and that’s converting this archive to a static website.</p><p>The best way to make this conversion is directly with the docc command line tool built into Xcode. It&#39;s usually found at the following location within Xcode.app: Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/docc, but it&#39;s much simpler to just run xcrun --find docc to find the actual executable. docc has command process-archive and transform-for-static-hosting, which are the main things we need. Other than that, we need to add the path to the doccarchive, hosting base path, and output path.</p><pre>$(xcrun --find docc) process-archive transform-for-static-hosting \<br>  /path/to/MyPackage.doccarchive --output-path docs;</pre><p>A new folder similar to this one will be created in the root of your project:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/390/1*nA4EJi4qBmXR3mTBEtbjcQ.png" /></figure><p>Now locally host the static site. To do this, go to the docs directory in Terminal and run python3 -m http.server.</p><p>Your documentation is now be visible at http://localhost:8000/documentation/PACKAGENAME. If you instead go to <a href="http://localhost:8000/documentation">http://localhost:8000/documentation</a>, it will list the package there for you to click through to.</p><p>If you are only hosting a single framework, it might make sense to replace index.html with a short script to redirect the user, such as:</p><pre>echo &quot;&lt;script&gt;window.location.href += \&quot;/documentation/packagename\&quot;&lt;/script&gt;&quot; &gt; docs/index.html</pre><blockquote>Note that the package name must be lowercase here. If you’re not sure, check the contents of docs/documentation/. In my example I would use realityui.</blockquote><p>This is an example of the resulting page that will be rendered:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CQEnLTvLMcLNEpJKdcY9pg.png" /></figure><h3>Hosting Swift Package Docs on GitHub</h3><p>For any repository on GitHub, you can create a static website that has the URL format:</p><pre>https://&lt;org-or-username&gt;.github.io/&lt;repo-name&gt;</pre><p>Hosting at this location works by telling GitHub where to fetch the website contents from, specifying the branch and directory. This gives a perfect place to host documentation for any Swift package.</p><p>One change we need to make before hosting these files, is adding a --hosting-base-path parameter to the Terminal command. The hosting base path needs to be the same as your repository name, as this is where GitHub hosts your static website.</p><pre>$(xcrun --find docc) process-archive \<br>    transform-for-static-hosting /path/to/MyPackage.doccarchive \<br>    --hosting-base-path REPONAME \<br>    --output-path docs;</pre><p>Now all you need to do is push that docs directory to your repository on GitHub and head to the repository settings to tell GitHub where your static site is.</p><p>In many repositories, this is on an orphan branch called gh-pages where only the site exists at the root directory.</p><p>gh-pages branch:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_BqK5k9y2bhih1tu8Ge0Ew.png" /></figure><p>GitHub repository settings:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GWqZ8NSYssfPPi7X18hQZA.png" /></figure><h3>Deploying with GitHub Actions</h3><p>Now that you understand how to deploy documentation in Apple’s DocC format on GitHub pages, you won’t exactly want to re-create this manually and launch your static website every time you make a new commit or merge in a pull request.</p><p>This is why it’s highly recommended that you turn your deployment into a <a href="https://github.com/features/actions">GitHub Action</a>!</p><p>You can make your GitHub Action directly upload and host your static website with a feature that’s currently in beta:</p><p><a href="https://github.blog/changelog/2022-07-27-github-pages-custom-github-actions-workflows-beta/">GitHub Pages: Custom GitHub Actions Workflows (beta) | GitHub Changelog</a></p><h3>GitHub Actions?</h3><p>To quickly summarise, GitHub Actions is an amazing CI/CD tool from GitHub. These can be used to run tests on your code. Actions can be executed base on pull requests, every time you push to a specific branch, create a new release, or even a cron job that executes periodically. Actions can be run on a number of different runner images, including macos, which we will use for this project. Actions are free for public repositories, and have very generous usage for private repositories.</p><p>If you haven’t used GitHub actions before, I’d highly recommend taking a <a href="https://github.com/features/actions">deeper look into it on GitHub</a>.</p><h3>Creating The Action</h3><p>First, head back to the settings of your repository and set the build and deployment source as “GitHub Actions”:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DNLJ13CFvHn6-X6b4f4pHg.png" /></figure><blockquote>This is a feature that’s <a href="https://github.blog/changelog/2022-07-27-github-pages-custom-github-actions-workflows-beta/">currently in beta</a>, but that doesn’t mean we can’t use it!</blockquote><p>As with every GitHub Action, create a YAML file inside the following repository at the root of your repo: .github/workflows. For example, .github/workflows/publish-pages.yml.</p><p>For most of the action, the setting will be fairly trivial and can be modified as you see fit. Here’s an example setup, up to where the jobs are defined:</p><pre><em># Name your workflow.</em><br>name: Deploy DocC</pre><pre>on:<br>  <em># Runs on pushes targeting the default branch</em><br>  push:<br>    branches: [&quot;main&quot;]</pre><pre><em># Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages</em><br>permissions:<br>  contents: read<br>  pages: write<br>  id-token: write</pre><pre><em># Allow one concurrent deployment</em><br>concurrency:<br>  group: &quot;pages&quot;<br>  cancel-in-progress: true</pre><p>From there we need to create a job that does several things:</p><ul><li>Checkout the repository.</li><li>Build DocC documentation, as above.</li><li>Upload the website as an artifact.</li><li>Publish that artifact got GitHub Pages.</li></ul><p>As this is building DocC, we will run this job on the macos-12 image that is provided. This is what the resulting job will look like:</p><pre>jobs:<br>  <em># Single deploy job since we&#39;re just deploying</em><br>  deploy:<br>    environment:<br>      <em># Must be set to this for deploying to GitHub Pages</em><br>      name: github-pages<br>      url: ${{ steps.deployment.outputs.page_url }}<br>    runs-on: macos-12<br>    steps:<br>      - name: Checkout 🛎️<br>        uses: actions/checkout@v3<br>      - name: Build DocC<br>        run: |<br>          xcodebuild docbuild -scheme MyPackage \<br>            -derivedDataPath /tmp/docbuild \<br>            -destination &#39;generic/platform=iOS&#39;;<br>          $(xcrun --find docc) process-archive \<br>            transform-for-static-hosting /tmp/docbuild/Build/Products/Debug-iphoneos/MyPackage.doccarchive \<br>            --hosting-base-path MyPackage \<br>            --output-path docs;<br>          echo &quot;&lt;script&gt;window.location.href += \&quot;/documentation/mypackage\&quot;&lt;/script&gt;&quot; &gt; docs/index.html;<br>      - name: Upload artifact<br>        uses: actions/upload-pages-artifact@v1<br>        with:<br>          <em># Upload only docs directory</em><br>          path: &#39;docs&#39;<br>      - name: Deploy to GitHub Pages<br>        id: deployment<br>        uses: actions/deploy-pages@v1</pre><blockquote><em>Remember to replace </em><em>MyPackage and </em><em>mypackage with your Swift Package name. And be sure to set the </em><em>derivedDataPath as above, otherwise there may be some caching issues on the deployment.</em></blockquote><p>That all looks quite big, but only really because of the multiple commands inside the Build DocC step. Since the end of September, the macos-12 image uses Xcode 14 as default, giving us the new DocC look.</p><p>The action <a href="https://github.com/actions/upload-pages-artifact">upload-pages-artifact</a> uploads the artifact with the name &quot;github-pages&quot;, and the action <a href="https://github.com/actions/deploy-pages">deploy-pages</a> looks for an artifact of the same name to then deploy.</p><p>If you head to the Actions tab of your repository and select the action that was just deployed, you should see that the action has succeeded like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*86m0TCaptrf681JiEuhGtg.png" /></figure><p>Upon clicking the displayed URL you will be redirected to your documentation page!</p><p>From now on, the documentation will automatically deploy whenever you push to the main branch of your repository; or upon another action, depending on how you’ve set it up.</p><h3>See an Example</h3><p>For a full example of this, check out <a href="https://github.com/AgoraIO-Community/VideoUIKit-iOS">VideoUIKit-iOS</a> repo, complete with the workflow file <a href="https://github.com/AgoraIO-Community/VideoUIKit-iOS/blob/main/.github/workflows/deploy_docs.yml">deploy_docs.yml</a>, and published documentation below. This package has a DocC documentation deployed in the same method as above, and also includes an article within DocC, with more updates to come.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ADx0BzFvBQe84iiqE-hl-Q.png" /></figure><h4>Current Drawbacks of DocC</h4><p>Currently, if your class has some helper methods extending classes found in other frameworks, such as Foundation, those methods will not be defined within the documentation.</p><p>This is being actively worked on by the community, so expect it to be included in future versions of Swift and Xcode:</p><p><a href="https://forums.swift.org/t/document-extensions-to-external-types-using-docc/56060">Document Extensions to External Types Using DocC</a></p><h3><strong>Conclusion</strong></h3><p>By following these steps, you should now be able to have your own deployment of DocC docs on your own Swift Package, or other project automatically on GitHub. Please leave a comment below with your package!</p><p>If you found this helpful, please feel free to drop some claps and a follow. 👏👏👏</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=218c5ca6cad5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Create a SwiftUI Streaming App with Fun Virtual Backgrounds]]></title>
            <link>https://medium.com/agora-io/create-a-swiftui-streaming-app-with-fun-virtual-backgrounds-a9c0736236a5?source=rss-2f5df90bc0eb------2</link>
            <guid isPermaLink="false">https://medium.com/p/a9c0736236a5</guid>
            <category><![CDATA[video-conferencing]]></category>
            <category><![CDATA[ios]]></category>
            <category><![CDATA[virtual-background]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[agora]]></category>
            <dc:creator><![CDATA[Max Cobb]]></dc:creator>
            <pubDate>Thu, 31 Mar 2022 16:36:44 GMT</pubDate>
            <atom:updated>2022-08-26T14:15:38.269Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Extub09Cnr3U1Zq0c1cSYQ.png" /></figure><p>Creating a video streaming app is super straightforward with the Agora SDK and Agora UIKit. But what about adding cool features to enhance the overall user experience or adding monetisation options? That’s where the Agora <a href="https://console.agora.io/marketplace">Extensions Marketplace</a> comes in.</p><p>Launched in September 2021, the Agora Extensions Marketplace is a one-stop-shop for app enrichment features like background and foreground segments, voice changers, and much more.</p><p>In this post, we’ll show you how to add or change eye-catching backgrounds to your streaming app with the VisionLab DoreSegment extension. With the<a href="https://console.agora.io/marketplace/extension/introduce?serviceName=visionlab"> DoreSegment</a> extension, you can get rid of background noise in three easy steps:</p><ol><li>Enable the extension.</li><li>Pass extension credentials.</li><li>Pass an image as the new background.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/473/1*ber-dJ-J2TH6eOQYqRS5dg.gif" /></figure><p>Now let’s get started</p><h3>Prerequisites</h3><ul><li><a href="https://sso.agora.io/en/signup?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=create-a-swiftui-streaming-app-with-virtual-backgrounds">An Agora developer account — Sign up here</a></li><li>Xcode 12.3 or later</li><li>A physical iOS device with iOS 13.0 or later</li><li>A basic understanding of iOS development</li></ul><h3>Setup</h3><p>Let’s start with a new, single-view iOS project. Create the project in Xcode, choosing SwiftUI for the interface, and then add the <a href="https://github.com/AgoraIO-Community/iOS-UIKit">Agora UIKit</a> package.</p><p>Add the package by selecting File &gt; Swift Packages &gt; Add Package Dependency, and then paste the link into this Swift package:</p><pre><a href="https://github.com/AgoraIO-Community/VideoUIKit-iOS.git">https://github.com/AgoraIO-Community/VideoUIKit-iOS.git</a></pre><p>When asked for the version, insert 4.0.0-preview, and select &quot;Up to Next Major&quot;. This should install at least version 4.0.0-preview.8, which is released at the time of writing.</p><p>When in doubt, you can always select “Exact” and insert 4.0.0-preview.8.</p><p>Once that package is installed, the camera and microphone usage descriptions need to be added. To see how to do that, check out Apple’s documentation here:</p><p><a href="https://developer.apple.com/documentation/avfoundation/cameras_and_media_capture/requesting_authorization_for_media_capture_on_ios#2962313">Apple Developer Documentation</a></p><h3>Video Call UI</h3><p>I won’t go into much detail about the UI here, because we’re mainly focusing on the background-replacing extension, but on a high level:</p><ul><li>The ContentView contains two views: an AgoraViewer (defined in <a href="https://github.com/AgoraIO-Community/iOS-UIKit">Agora UIKit</a>) and a button to join the channel.</li><li>When the join button is pressed, the Agora UIKit method join(channel:,with:,as:) is called, which joins the Agora video channel.</li></ul><p>This is what the ContentView looks like:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/55f164fcb0f5fada6b98dc430b3d3f82/href">https://medium.com/media/55f164fcb0f5fada6b98dc430b3d3f82/href</a></iframe><p>The AgoraViewerHelper helper object, without any of the DoreSegment additions looks like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cf8c350c31a68c1346111a29efde8fcc/href">https://medium.com/media/cf8c350c31a68c1346111a29efde8fcc/href</a></iframe><p>In the above snippet, we have added a button with a magic wand image. When clicked, it will call the method, toggleBackground.</p><p>We now have a working video call app with SwiftUI and Agora UIKit. The next step is to integrate the DoreSegment extension!</p><h3>Integrating DoreSegment</h3><h4>Credentials</h4><p>If you have an account with Agora and are currently signed in, follow <a href="https://console.agora.io/marketplace/extension/introduce?serviceName=visionlab">this link</a> to activate the extension for your account.</p><p>First, you will need to activate DoreSegment for your account by clicking “Activate” here:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eWoZ0zgvMDSZpJV4f1CC5Q.png" /></figure><p>Next, enable DoreSegment for the project, the same project as for the Agora App ID used in the app:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QWhsmNyUpST1gZgrYRGGGQ.png" /></figure><p>Once that’s done, you can grab your DoreSegment API Key and API Secret by clicking “view” below the Credentials column:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0l7FcR7RJdUimovbwfc7YA.png" /></figure><h4>Add to Xcode</h4><p>We need to add the extension, which can also be installed via the Swift Package Manager with the following URL:</p><p><a href="https://github.com/AgoraIO-Community/Extension-VisionLab-DoreSegment-iOS.git">https://github.com/AgoraIO-Community/Extension-VisionLab-DoreSegment-iOS.git</a></p><blockquote><em>The latest release at the time of writing is </em><em>1.0.1.</em></blockquote><p>This package doesn’t need importing into your Swift code. It just needs to be bundled alongside the Agora SDK in your app target.</p><h3>Enable the Extension</h3><p>When using Agora extensions, you need to enable them before joining a channel.</p><p>To do this with Agora UIKit, you can call AgoraVideoViewer.enableExtension().</p><p>This method needs to be passed the vendor (“DoreAI”) and the vendor extension (“DoreSegment”) in this case:</p><pre>AgoraViewerHelper.agview.viewer.enableExtension(<br>  withVendor: &quot;DoreAI&quot;, extension: &quot;DoreSegment&quot;, enabled: true<br>)</pre><p>The above code snippet is added to the beginning of the joinChannel method defined earlier.</p><p>After joining the channel, we are going to disable the extension again because we want the feature to be disabled until someone clicks the button to set a new background. But the feature must be enabled once before joining the channel, so we must enable and then disable it:</p><pre>func joinedChannel(channel: String) {<br>  registerDoreSegment()<br>​<br>  AgoraViewerHelper.agview.viewer.enableExtension(<br>    withVendor: &quot;DoreAI&quot;, extension: &quot;DoreSegment&quot;, enabled: false<br>  )<br>}</pre><h3>Toggle the Extension</h3><p>As shown in the above snippet for AgoraViewerHelper, we have added an extra button to the default buttons provided by Agora UIKit.</p><p>This button displays the magic wand icon and toggles between a green and a grey background. When clicked, the button calls a method toggleBackground. Let’s link up this button to enable and disable the extension, and apply an image as the background image (bgImage):</p><pre>@objc func toggleBackground(_ sender: UIButton) {<br>  virtualBackgroundEnabled.toggle()<br>  sender.backgroundColor = self.virtualBackgroundEnabled ?<br>    .systemGreen : .systemGray<br>  if self.virtualBackgroundEnabled {<br>    let backgroundImg = UIImage(named: &quot;background-boat&quot;)<br>    guard let base64Img = backgroundImg?.pngData()<br>      ?.base64EncodedString()<br>    else {<br>     fatalError(&quot;Background image not found in bundle&quot;)<br>    }<br>    AgoraViewerHelper.agview.viewer.setExtensionProperty(<br>      &quot;DoreAI&quot;, extension: &quot;DoreSegment&quot;,<br>      key: &quot;bgImage&quot;, value: base64Img<br>    )<br>  }<br>  AgoraViewerHelper.agview.viewer.enableExtension(<br>    withVendor: &quot;DoreAI&quot;, extension: &quot;DoreSegment&quot;,<br>    enabled: self.virtualBackgroundEnabled<br>  )<br>}</pre><p>In the above snippet, we fetch an image from the app&#39;s bundle named background-boat if the background is now enabled. That image is passed through to DoreSegment so that it knows what image we want as the background. Then we call enableExtension again, either enabling or disabling the extension. The image used in the background is included in the GitHub project but can also be downloaded <a href="https://github.com/AgoraIO-Community/VisionLab-DoreSegment-SwiftUI/raw/main/VisionLab-DoreSegment-SwiftUI/Assets.xcassets/background-boat.imageset/background-boat.jpg">here</a>.</p><p>Almost there now! The only thing left to do is define the registerDoreSegment method.</p><p>This method needs to pass the API Key and API Secret values. These values need to be passed to the extension with the keys apiKey and license respectively:</p><pre>func registerDoreSegment() {<br>  // Set API Credentials<br>  let doreCredentials: [String: String] = [<br>    &quot;apiKey&quot;: AppKeys.visionLabApiKey,<br>    &quot;license&quot;: AppKeys.visionLabApiSecret<br>  ]<br>  AgoraViewerHelper.agview.viewer.setExtensionProperty(<br>    &quot;DoreAI&quot;, extension: &quot;DoreSegment&quot;,<br>    key: &quot;start&quot;, codable: doreCredentials<br>  )<br>}</pre><p>This is the final result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/473/1*ber-dJ-J2TH6eOQYqRS5dg.gif" /></figure><h3>Conclusion</h3><p>Congratulations! You now have a video call app complete with the new extension from VisionLab!</p><p>There are other extensions I’d encourage you to try out. They can all be found here:</p><p><a href="https://console.agora.io/marketplace">Dashboard</a></p><p>If you want to create an app from scratch with this extension, the extension can be applied to an application made directly with the Agora SDK. For that, you will need to make sure you’re using the 4.x version of the SDK. Details can be found here:</p><p><a href="https://docs.agora.io/en/video-call-4.x-preview/landing-page?platform=iOS">Video Call v4.0.0 Preview</a></p><p>The methods, such as setExtensionProperty, used in this post are adaptations of the SDK&#39;s built-in methods relating to extensions. They can be found in this file from Agora Video UIKit:</p><p><a href="https://github.com/AgoraIO-Community/iOS-UIKit/blob/4.0.0-preview.8/Sources/Agora-UIKit/AgoraVideoViewer%2BAgoraExtensions.swift">iOS-UIKit/AgoraVideoViewer+AgoraExtensions.swift at 4.0.0-preview.8 · AgoraIO-Community/iOS-UIKit</a></p><h3>Testing</h3><p>You can try out this app by following this GitHub link:</p><p><a href="https://github.com/AgoraIO-Community/VisionLab-DoreSegment-SwiftUI">GitHub - AgoraIO-Community/VisionLab-DoreSegment-SwiftUI</a></p><h3>Other Resources</h3><p>For more information about building applications using Agora SDKs, take a look at the <a href="https://docs.agora.io/en/Video/start_call_ios?platform=iOS&amp;utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=create-a-swiftui-streaming-app-with-virtual-backgrounds">Agora Video Call Quickstart Guide</a> and <a href="https://docs.agora.io/en/Video/API%20Reference/oc/docs/headers/Agora-Objective-C-API-Overview.html?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=create-a-swiftui-streaming-app-with-virtual-backgrounds">Agora API Reference</a>.</p><p>I also invite you to <a href="https://www.agora.io/en/join-slack/">join the Agora Developer Slack community</a> to meet with our developer team as well as other like-minded developers and technology enthusiasts.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a9c0736236a5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/agora-io/create-a-swiftui-streaming-app-with-fun-virtual-backgrounds-a9c0736236a5">Create a SwiftUI Streaming App with Fun Virtual Backgrounds</a> was originally published in <a href="https://medium.com/agora-io">Agora.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Universal Links and SwiftUI Video Calls]]></title>
            <link>https://maxxfrazer.medium.com/universal-links-and-swiftui-video-calls-704abfc80bf0?source=rss-2f5df90bc0eb------2</link>
            <guid isPermaLink="false">https://medium.com/p/704abfc80bf0</guid>
            <category><![CDATA[agora]]></category>
            <category><![CDATA[deep-linking]]></category>
            <category><![CDATA[ios]]></category>
            <category><![CDATA[universal-links]]></category>
            <category><![CDATA[swiftui]]></category>
            <dc:creator><![CDATA[Max Cobb]]></dc:creator>
            <pubDate>Tue, 15 Mar 2022 12:11:56 GMT</pubDate>
            <atom:updated>2022-08-26T14:21:26.401Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YqO8d54VL30yfYrq16IPQg.jpeg" /></figure><p>When making an application such as one for video calls, you may want to share links to let your users’ friends jump into the same video call or live stream that they’re watching. The second-best way to achieve this is by sharing a room code for people to type in to their version of the app. But a much better option is to share a link that opens the app right to the place you want. That is where we can use universal links.</p><h3>What are Universal Links?</h3><p>Often, if there is an equivalent web page for a view that exists in an app. For example, if you’re viewing an Instagram profile on your phone’s web browser, there is also a way to show the same content in the Instagram app. Universal links are the gateway for going from the web browser content to the app’s equivalent content.</p><p>In some cases, universal links can’t have a web equivalent; for example, a complex game that will not run in a web browser. A universal link in that scenario could lead to a template page, which can then be passed through to your application to render the full content or join the game.</p><p>Check out Apple’s documentation for more information on universal links:</p><p><a href="https://developer.apple.com/ios/universal-links/">Universal Links - Apple Developer</a></p><p>Universal Links also work on Android devices, but the setup on your website as well as the device is a little different.</p><h3>Prerequisites</h3><ul><li><a href="https://sso.agora.io/en/signup?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=universal-links-and-swiftui-video-calls">An Agora developer account — Sign up here</a></li><li>Xcode 12.3 or later</li><li>A physical iOS device with iOS 13.0 or later</li><li>A public server with an SSL certificate (https)</li></ul><h3>Setting Up Universal Links</h3><p>Setting up universal links has been made easier in recent years, but it still has two main steps:</p><ol><li>Setting Up the Website</li><li>Setting Up the App</li></ol><p>Let’s start with the website side.</p><h3>Setting Up the Website</h3><p>This section is probably the easiest part of universal links. All you need to do is add a file named apple-app-site-association, which contains the correct values for your application:</p><pre>{<br>  &quot;applinks&quot;: {<br>    &quot;apps&quot;: [],<br>    &quot;details&quot;: [{<br>        &quot;appIDs&quot;: [&quot;DEVELOPMENT_TEAM.PRODUCT_BUNDLE_IDENTIFIER&quot;],<br>        &quot;components&quot;: [<br>          {<br>            &quot;/&quot;: &quot;*&quot;<br>          }<br>        ]<br>    }]<br>  }<br>}</pre><p>The above example will put a universal link button on every page of your website. See below for how to specify specific URLs in your domain.</p><p>You just need to replace DEVELOPMENT_TEAM and PRODUCT_BUNDLE_IDENTIFIER with the associated values from your app. To find them, you can run a couple of commands in a Terminal from your project’s root folder, such as this:</p><pre>grep -r -m 1 &quot;DEVELOPMENT_TEAM&quot; .; grep -r -m 1 &quot;PRODUCT_BUNDLE_IDENTIFIER&quot; .</pre><p>Your terminal will look similar to this:</p><pre>=&gt; grep -r -m 1 &quot;DEVELOPMENT_TEAM&quot; .; grep -r -m 1 &quot;PRODUCT_BUNDLE_IDENTIFIER&quot; .</pre><pre>./AppName.xcodeproj/project.pbxproj:    DEVELOPMENT_TEAM = 278494H572;<br>./AppName.xcodeproj/project.pbxproj:    PRODUCT_BUNDLE_IDENTIFIER = io.agora.AppName;</pre><p>In this case, the values we want are 278494H572 and io.agora.AppName, respectively.</p><p>To check that your website is set up correctly, head to <a href="https://branch.io/resources/aasa-validator/">branch.io’s validator</a>. One common issue is that the content-type header cannot be found. If you face this issue, you will need to create or add these lines to .htaccess at the same location as the apple-app-site-association file:</p><pre>&lt;Files &quot;apple-app-site-association&quot;&gt;<br>  ForceType &#39;application/json&#39;<br>&lt;/Files&gt;</pre><p>With the current setup, all pages on the website will link through to the specified app. If you just want a specific part of your website to link to an app, you can modify the components array object or add another one. For example, if the current components array is replaced with:</p><pre>&quot;components&quot;: [<br>  { &quot;/&quot;: &quot;/wwdc/news&quot; },<br>  { &quot;/&quot;: &quot;/videos/wwdc/2015/*&quot; }<br>]</pre><p>Then all the WWDC news articles would direct to the app, as well as all videos from 2015.</p><p>More examples can be found in Apple’s documentation <a href="https://developer.apple.com/documentation/xcode/supporting-associated-domains">here</a>.</p><p>For our app, we are going to add the following component:</p><pre>{<br>  &quot;/&quot;: &quot;/join*&quot;,<br>  &quot;?&quot;: { &quot;channel&quot;: &quot;?*&quot; }<br>}</pre><p>Using example.com as our domain address, this is what an example of the full URL might look like:</p><p>https://www.example.com/join?channel=dkvn3lw2</p><p>For more information about how to set up universal links with an apple-app-site-association file, Apple has a great resource from WWDC 2019 <a href="https://developer.apple.com/videos/play/wwdc2019/717">here</a>.</p><p>The full file should look like this:</p><pre>{<br>  &quot;applinks&quot;: {<br>    &quot;details&quot;: [<br>      {<br>        &quot;appIDs&quot;: [&quot;DEVELOPMENT_TEAM.PRODUCT_BUNDLE_IDENTIFIER&quot;],<br>        &quot;components&quot;: [<br>          {<br>            &quot;/&quot;: &quot;/join/*&quot;,<br>            &quot;?&quot;: {<br>              &quot;channel&quot;: &quot;?*&quot;<br>            }<br>          }<br>        ]<br>      }<br>    ]<br>  }<br>}</pre><p>Now that that’s set up, let’s get to the app portion of our app.</p><h3>Setting Up the App</h3><h4>Installing the Package</h4><p>Set up a new SwiftUI app in Xcode, and add the following Swift package to your project with this URL:</p><p><a href="https://github.com/AgoraIO-Community/VideoUIKit-iOS.git">https://github.com/AgoraIO-Community/VideoUIKit-iOS.git</a></p><blockquote><em>The current stable release of Agora UIKit is 1.7.1</em></blockquote><p>If you’re not sure how to add a Swift package, Apple’s documentation has it covered with clear steps:</p><p><a href="https://developer.apple.com/documentation/swift_packages/adding_package_dependencies_to_your_app">Apple Developer Documentation</a></p><h4>Setting the Views</h4><p>We’ll get this part out of the way quickly, as we want to get back to the universal links part of this guide. The first thing we need to do is add a button to create and join a channel:</p><pre>Button {<br>  // click action here<br>} label: {<br>  Text(&quot;Create Channel&quot;)<br>    .padding().background(.green).cornerRadius(30)<br>}​</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/559/1*ym_cArinN0cFSyOxduKpPA.jpeg" /></figure><p>This button’s action should create a channel name and display an AgoraViewer. AgoraViewer is a subclass of View that is provided by Agora UIKit. This can be achieved with the following block of code:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/db7d28fd865baa7fca64e97db998a3f2/href">https://medium.com/media/db7d28fd865baa7fca64e97db998a3f2/href</a></iframe><p>Above, the channel is set to a static string &quot;test&quot;, and the property isShowingVideo is set to true. The &quot;test&quot; string will be replaced with a random string generator in the final product on GitHub.</p><p>isShowingVideo is a boolean which is used to decide if the video call view (videoCallView) should be presented or not by the NavigationView. The next things to add are quite straightforward, we need to join and leave the channel using the onAppear and onDisappear methods, and add some buttons to the top of the camera view.</p><pre>.onAppear(perform: {<br>  guard let channelName = self.channelName else {<br>    self.isShowingVideo = false<br>    return<br>  }<br>  ContentView.agview.join(<br>    channel: channelName, with: nil, as: .broadcaster<br>  )<br>})<br>.onDisappear(perform: {<br>  ContentView.agview.viewer.leaveChannel()<br>})</pre><p>If channelName is nil when the video view is appearing then we will close it again, this is because a channel name is required, so something must have gone wrong getting to this stage. After that we call the builtin function from Agora UIKit to join the channel as a broadcaster.</p><p>The second parameter in the join method is for a token, which I am not using in my test application, but is required for production apps that use Agora.</p><p>To find out more about working with tokens using Agora’s SDKs, here are a few resources:</p><ul><li><a href="https://docs.agora.io/en/Interactive%20Broadcast/token_server">Authenticate Your Users with Tokens</a></li><li><a href="https://www.agora.io/en/blog/connecting-to-agora-with-tokens-using-swift/">Connecting to Agora with Tokens</a></li></ul><p>Now the final part of the UI is setting up the other buttons in that videoCallView above.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/eef599a0df5d7f8aacbe7ee38cd66f00/href">https://medium.com/media/eef599a0df5d7f8aacbe7ee38cd66f00/href</a></iframe><p>Two buttons have been added in the above code snippet. Since they’re positioned in an HStack, they appear side by side, with a Spacer() in between.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*xoDFCPJ6n3NZPk8XGKAOsg.jpeg" /></figure><p>The first button lets the user exit the current view, leaving the call and going back to the main screen. The second button creates a URL (using example.com as the domain) with the channel as a parameter. This generated URL will open up the default share screen, to send it natively via iMessage, AirDrop, email, etc.</p><p>The share screen looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sTYhwWBmG9ln_inndcIsrw.jpeg" /></figure><h3>Interpreting the Universal Link</h3><p>To let your app interpret a Universal link, your app must know which domains it is allowed to talk to. This is a very straightforward project, and can now be done directly with Xcode.</p><p>Head to the Signing &amp; Capabilities section of your app within Xcode, and add the Associated Domains capability like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q_5_WU4y_0lPbqBKPTMEFg.png" /></figure><p>Once that’s done, set the domains within the new capability, here:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yMN3UnbKIvraFCllGK8aSA.png" /></figure><p>The above entries say applinks:example.com and applinks:www.example.com. Replace “example.com” with your domain that is hosting the apple-app-site-association.</p><p>The final step in this application is understanding when the app has been opened from a universal link, and getting the full URL used to open the app.</p><p>SwiftUI offers a very easy to implement method for capturing universal links that have opened the app, which is the <a href="https://developer.apple.com/documentation/swiftui/view/onopenurl(perform:)">onOpenURL</a> method. This method can be appended to any <a href="https://developer.apple.com/documentation/swiftui/view">View</a>, and its only parameter is the URL object. We can attach this to our NavigationView:</p><pre>NavigationView {<br>  // Navigation View content here<br>}.onOpenURL { url in<br>  print(url.absoluteString)<br>}</pre><p>When opening our app via the universal link, the above code block will execute and print our full URL string.</p><p>All we really need from this is that channel parameter, the final part of the URL.</p><p>To extract parts of the URL into a dictionary, I am calling a property called queryDictionary, which has been grabbed from the following Stack Overflow answer:</p><p><a href="https://stackoverflow.com/a/46603619/2156765">How do I convert url.query to a dictionary in Swift?</a></p><p>Using queryDictionary, we can get our channel name, and once again trigger the video feed to open as so:</p><pre>if let channel = url.queryDictionary?[&quot;channel&quot;] {<br>  self.channelName = channel<br>  self.isShowingVideo = true<br>}</pre><p>The app is complete! Now when going to your domain address with the endpoint join?channel=test, it will launch your application and go straight to that channel name.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/443/1*ICWVjZgscb3ht7mz_OuMWQ.gif" /></figure><h3>Testing</h3><p>This entire project is available on GitHub here:</p><p><a href="https://github.com/AgoraIO-Community/Universal-Links-SwiftUI">GitHub - AgoraIO-Community/Universal-Links-SwiftUI</a></p><p>When running the application, be sure to head over to the Signing and Capabilities section to update the domain (it will be set to example.com by default). Also once changing the Bundle and Team IDs, run the following command in Terminal to get your new values:</p><pre>grep -r -m 1 &quot;DEVELOPMENT_TEAM&quot; .; grep -r -m 1 &quot;PRODUCT_BUNDLE_IDENTIFIER&quot; .</pre><h3>Other Resources</h3><p>For more information about building applications using Agora SDKs, take a look at the <a href="https://docs.agora.io/en/Video/start_call_ios?platform=iOS&amp;utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=universal-links-and-swiftui-video-calls">Agora Video Call Quickstart Guide</a> and <a href="https://docs.agora.io/en/Video/API%20Reference/oc/docs/headers/Agora-Objective-C-API-Overview.html?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=universal-links-and-swiftui-video-calls">Agora API Reference</a>.</p><p>I also invite you to <a href="https://www.agora.io/en/join-slack/">join the Agora Developer Slack community</a> to meet with our developer team as well as other like-minded developers and technical enthusiasts.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=704abfc80bf0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Agora Releases Native SDK v3.6.2]]></title>
            <link>https://medium.com/agora-io/agora-releases-native-sdk-v3-6-2-1cf8c735ef17?source=rss-2f5df90bc0eb------2</link>
            <guid isPermaLink="false">https://medium.com/p/1cf8c735ef17</guid>
            <category><![CDATA[agora]]></category>
            <category><![CDATA[release-notes]]></category>
            <category><![CDATA[sdk]]></category>
            <category><![CDATA[swift-package-manager]]></category>
            <category><![CDATA[video-communication]]></category>
            <dc:creator><![CDATA[Max Cobb]]></dc:creator>
            <pubDate>Wed, 23 Feb 2022 12:28:19 GMT</pubDate>
            <atom:updated>2022-02-25T20:33:56.299Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>Voice and Video SDK v3.6.2 now available for on all supported native mobile and desktop platforms. See what’s new below!</em></h4><figure><img alt="Agora release banner" src="https://cdn-images-1.medium.com/max/1024/1*JpiEB3ZO16jS_oCNU5en-g.jpeg" /></figure><blockquote>Agora Native SDK 3.6.2 was released on 22nd February 2022.</blockquote><h3>Compatibility Changes ⚙️</h3><h4>Dynamic Libraries (macOS)</h4><p>This update brings the RTM libraries for macOS to XCFramework format. This improves compatibility with different types of Macs (M1), and means that the library is now availabe via Swift Package Manager:</p><p><a href="https://github.com/AgoraIO/AgoraRtcEngine_macOS">GitHub - AgoraIO/AgoraRtcEngine_macOS</a></p><h3>New Features ✨</h3><h4>Video Enhancement 🪄</h4><p>Several new video enhancement properties are now available on all supported platforms:</p><ul><li>Video noise reduction</li><li>Low-light enhancement</li><li>Color enhancement</li></ul><p>The associated libraries for these features are as follows:</p><p><strong>Android</strong>: libagora_video_process_extension.so<br><strong>iOS + macOS</strong>: AgoraVideoProcessExtension.xcframework<br><strong>Windows</strong>: libagora_video_process_extension.dll</p><h4>Loading .so Files Dynamically (Android)</h4><p>A new method setAgoraLibPath has been added, which allows you to dynaimcally load a .so file based on your specified directory when the app is running.</p><p>This can reduce the size of your app! For more information, see this resource:</p><p><a href="https://docs.agora.io/en/Video/faq/reduce_app_size_rtc">How can I reduce the app size after integrating the RTC Native SDK?</a></p><h4>Passing in native TextureView (Android)</h4><p>This release adds support for directly using Android native TextureView.</p><h3>Improvements 🛠</h3><h4>Audio Device Test (macOS/Windows) 🔊</h4><p>A new function has been added, onAudioDeviceTestVolumeIndication, which reports the volume information of the local audio device being tested.</p><p>Additionally the other audio device tests can now be run while in a call, as well as before joining a call.</p><h4>Audio Recording 🎙</h4><p>This release extends startAudioRecording with support for setting recording with dual channels and higher audio quality.</p><h4>Screen sharing (macOS) 🖥</h4><p>This release optimizes the screen capture strategy and increases the upper limit of the screen capture frame rate.</p><h4>Music File Playback 🎼</h4><p>This release optimizes the experience of calling startAudioMixing to play music files by:</p><ul><li>Reduces issues of audio freezes or no audio when playing music files.</li><li>The audio duration reported by getAudioFileInfo is more accurate.</li><li>Supports more audio file formats. See <a href="https://docs.agora.io/en/Video/faq/audio_format">What formats of audio files does the Agora RTC SDK support</a>.</li></ul><h4>Audio file formats (iOS/macOS) 📂</h4><p>This release adds support for playing mono-track mp4 files, and multi-track online files in m4a and mp4 formats.</p><h4>Cloud Proxy ☁️</h4><p>To enrich application scenarios of the Agora Cloud Proxy, this release updates the cloud proxy types adds a TCP encrypted mode. In this mode, the SDK always transmits data over TCP/TLS 443.</p><p>These are just a handful of the updates to the SDK in v3.6.0. To see the full release notes for the native platform you’re using, check out one of the following links: <a href="https://docs.agora.io/en/Video/release_ios_video?platform=iOS#v362">iOS</a>, <a href="https://docs.agora.io/en/Video/release_android_video?platform=Android#v362">Android</a>, <a href="https://docs.agora.io/en/Video/release_mac_video?platform=macOS#v362">macOS</a>, and <a href="https://docs.agora.io/en/Video/release_windows_video?platform=Windows#v362">Windows</a>. Head to Agora’s <a href="https://docs.agora.io/en/Video/downloads?platform=All%20Platforms">downloads page</a> to get the latest SDK now.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1cf8c735ef17" width="1" height="1" alt=""><hr><p><a href="https://medium.com/agora-io/agora-releases-native-sdk-v3-6-2-1cf8c735ef17">Agora Releases Native SDK v3.6.2</a> was originally published in <a href="https://medium.com/agora-io">Agora.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Quickstart with Agora UIKit for iOS]]></title>
            <link>https://medium.com/agora-io/quickstart-with-agora-uikit-for-ios-f4ecd04ed8d7?source=rss-2f5df90bc0eb------2</link>
            <guid isPermaLink="false">https://medium.com/p/f4ecd04ed8d7</guid>
            <category><![CDATA[ios-app-development]]></category>
            <category><![CDATA[ios]]></category>
            <category><![CDATA[uikit]]></category>
            <category><![CDATA[live-streaming]]></category>
            <category><![CDATA[agora]]></category>
            <dc:creator><![CDATA[Max Cobb]]></dc:creator>
            <pubDate>Fri, 26 Nov 2021 18:33:52 GMT</pubDate>
            <atom:updated>2022-08-26T14:20:29.669Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9ZKPQYZDB6GYMDic23WoyA.png" /></figure><p>When adding a video chat to your own application, you might not want to read even more documentation. With Agora UIKit, you don’t have to think about much beyond what you already know about building familiar elements, such as a UIView.</p><p>This guide is a very short intro to Agora UIKit for iOS. To see the full guide, including code samples for macOS check it out here:</p><p><a href="https://www.agora.io/en/blog/how-to-create-ios-mac-os-video-streaming-app-3-lines-of-code/">iOS and macOS Video Streaming Application with a Few Lines of Code</a></p><p>The GitHub repository of this library can be found here:</p><p><a href="https://github.com/AgoraIO-Community/VideoUIKit-iOS.git">GitHub - AgoraIO-Community/VideoUIKit-iOS: Swift package and CocoaPod to simply integrate Agora Video Calling or Live Video Streaming to your iOS or macOS app with just a few lines of code.</a></p><h3>Prerequisites</h3><ul><li>An Agora developer account (see <a href="https://www.agora.io/en/blog/how-to-get-started-with-agora?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=ios-uikit-guide">How To Get Started with Agora</a>)</li><li>Xcode 11.0 or later</li><li>An iOS device with minimum iOS 13.0</li><li>A basic understanding of iOS development</li><li>CocoaPods (if not using Swift Package Manager)</li></ul><h3>Setup</h3><p>Start by creating an iOS or macOS project in Xcode.</p><h4>Swift Package Manager (Recommended, iOS Only)</h4><p>Add the URL of this repository to your Xcode 11+ Project.</p><p>Go to File &gt; Swift Packages &gt; Add Package Dependency, and paste in this link: <a href="https://github.com/AgoraIO-Community/VideoUIKit-iOS.git">https://github.com/AgoraIO-Community/VideoUIKit-iOS.git</a></p><p>When installing, make sure you select the AgoraUIKit checkbox when choosing the package products.</p><blockquote><em>Xcode sometimes has issues downloading large libraries such as the Agora SDK. If any errors are thrown, in Xcode select </em>File &gt; Swift Packages &gt; Reset Package Caches<em>.</em></blockquote><h4>CocoaPods</h4><p>To install the CocoaPod <a href="https://cocoapods.org/pods/AgoraUIKit_iOS">AgoraUIKit_iOS</a> or <a href="https://cocoapods.org/pods/AgoraUIKit_macOS">AgoraUIKit_macOS</a>, your Podfile should look like this:</p><pre>target &#39;Agora-UIKit-Project&#39; do<br>  pod &#39;AgoraUIKit_iOS&#39;, &#39;~&gt; 1.8.0&#39;<br>end</pre><blockquote>The latest AgoraUIKit release at the time of writing this post is v1.5.0</blockquote><p>Run pod init, and open the .xcworkspace file to get started.</p><p>Add authorisation for the app to use the camera and microphone. To do this, open up the Info.plist file at the root of your Xcode project and add <a href="https://developer.apple.com/library/archive/documentation/General/Reference/InfoPlistKeyReference/Articles/CocoaKeys.html#//apple_ref/doc/plist/info/NSCameraUsageDescription">NSCameraUsageDescription</a> along with <a href="https://developer.apple.com/library/archive/documentation/General/Reference/InfoPlistKeyReference/Articles/CocoaKeys.html#//apple_ref/doc/uid/TP40009251-SW25">NSMicrophoneUsageDescription</a>.</p><p>For more information on requesting authorisation for media capture, check out <a href="https://developer.apple.com/documentation/avfoundation/cameras_and_media_capture/requesting_authorization_for_media_capture_on_ios">this article from Apple</a>.</p><h3>Adding Video Streaming</h3><p>If you want a ViewController that simply loads a video feed, the entire ViewController can look like this:</p><pre>import UIKit<br>import AgoraUIKit_iOS<br>​<br>class ViewController: UIViewController {<br>  var agoraView: AgoraVideoViewer!<br>  override func viewDidLoad() {<br>    super.viewDidLoad()<br>    self.agoraView = AgoraVideoViewer(<br>      connectionData: AgoraConnectionData(<br>        appId: &lt;#Agora App ID#&gt;,<br>        appToken: &lt;#Agora Token or nil#&gt;<br>      )<br>    )<br>    // fill the view<br>    self.agoraView.fills(self.view)<br>    // join the channel &quot;test&quot;<br>    agoraView.join(channel: &quot;test&quot;, as: .broadcaster)<br>  }<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TfXi8nf4AMvrIN57VuQYPw.png" /></figure><p>That’s all that’s required when using Agora UIKit in the most basic way. However, this does not take tokens into consideration, and tokens are required in production apps using Agora for security.</p><h3>Token Integration</h3><p>Tokens are another step that can slow down launching a product with Agora. Several blogs and repositories are available for help with understanding tokens; including a<a href="https://github.com/AgoraIO-Community/agora-token-service"> prebuilt token server in Golang</a>, a<a href="https://medium.com/agora-io/2-click-setup-testing-token-server-b1c6fea9e1ad"> 2-Click setup guide</a> and information about<a href="https://www.agora.io/en/blog/connecting-to-agora-with-tokens-using-swift/"> fetching a token into your Swift app</a>.</p><p>With Agora UIKit, you can simply set the tokenURL to the location of your token server, and Agora UIKit will take care of the rest, assuming that it follows the endpoint structures shown in the<a href="https://github.com/AgoraIO-Community/agora-token-service"> Golang token server repository</a>. To do so, you will need to define an AgoraSettings object, assign the tokenURL value, and then apply the settings object when creating your AgoraVideoViewer object:</p><pre>let agoraSettings = AgoraSettings()<br>agSettings.tokenURL = &quot;https://&lt;heroku url&gt;&quot;<br>agoraView = AgoraVideoViewer(<br>  connectionData: AgoraConnectionData(<br>    appId: &lt;#Agora App ID#&gt;<br>  ),<br>  agoraSettings: agoraSettings<br>)</pre><p>Now, whenever you’re connecting to a new channel, use the join method that asks for a Boolean fetchToken value, and set that to true:</p><pre>agoraView.join(<br>  channel: &quot;test&quot;, as: .broadcaster, fetchToken: true<br>)</pre><h3>Summary</h3><p>Creating a production application that uses Agora’s network for video calling is now easier than ever. The same classes and methods seen above can also be added to a macOS application.</p><p>You can head over to the<a href="https://www.agora.io/en/blog/how-to-create-ios-mac-os-video-streaming-app-3-lines-of-code/"> Advanced Agora UIKit blog</a> to unlock the full potential of Agora UIKit for iOS and macOS. Many more customisations and enhancements can be made using this library.</p><h3>Other Resources</h3><p>For more information about building applications using Agora.io SDKs, take a look at the <a href="https://docs.agora.io/en/Video/start_call_ios?platform=iOS&amp;utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=real-time-messaging-video-dynamic-channels">Agora Video Call Quickstart Guide</a> and <a href="https://docs.agora.io/en/Video/API%20Reference/oc/docs/headers/Agora-Objective-C-API-Overview.html?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=real-time-messaging-video-dynamic-channels">Agora API Reference</a>.</p><p>I also invite you to <a href="https://www.agora.io/en/join-slack/">join the Agora.io Developer Slack community</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f4ecd04ed8d7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/agora-io/quickstart-with-agora-uikit-for-ios-f4ecd04ed8d7">Quickstart with Agora UIKit for iOS</a> was originally published in <a href="https://medium.com/agora-io">Agora.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Create a Voice Changing Video Call App with SwiftUI]]></title>
            <link>https://maxxfrazer.medium.com/create-a-voice-changing-video-call-app-with-swiftui-faefec5a564?source=rss-2f5df90bc0eb------2</link>
            <guid isPermaLink="false">https://medium.com/p/faefec5a564</guid>
            <dc:creator><![CDATA[Max Cobb]]></dc:creator>
            <pubDate>Fri, 19 Nov 2021 00:01:52 GMT</pubDate>
            <atom:updated>2022-08-26T14:23:47.799Z</atom:updated>
            <content:encoded><![CDATA[<p>Creating a video call app with Agora is super straightforward, but what about adding cool features to enhance the overall user experience or adding monetisation options? That’s where Agora’s <a href="https://console.agora.io/marketplace">Extensions Marketplace</a> comes in.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9G7gGW41h1vuglpb6tJSew.png" /></figure><p>The Extensions Marketplace launched last month, and already several extensions are available, with more to come.</p><p>In this post, we’ll show you how to easily add a voice changer to your existing video call app and choose between the available voice options. Thanks to the <a href="https://console.agora.io/marketplace/extension/introduce?serviceName=voicemod">Voicemod Voice Changer</a>, in just three easy steps you will be able to modify your voice with this extension:</p><ol><li>Enable the extension</li><li>Pass extension credentials</li><li>Choose the voice</li></ol><p>Now, let’s get started.</p><h4>Prerequisites</h4><ul><li><a href="https://sso.agora.io/en/signup?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=create-a-voice-changing-video-call-app-with-swiftui">An Agora Developer Account — Sign up here</a></li><li>Xcode 12.3 or later</li><li>A physical iOS device with iOS 13.0 or later</li><li>A basic understanding of iOS development</li></ul><h4>Setup</h4><p>Let’s start with a new, single-view iOS project. Create the project in Xcode, choosing SwiftUI for the interface, and then add the <a href="https://github.com/AgoraIO-Community/iOS-UIKit">Agora UIKit</a> package.</p><p>Add the package by selecting File &gt; Swift Packages &gt; Add Package Dependency, and then paste the link into this Swift package:</p><p><a href="https://github.com/AgoraIO-Community/VideoUIKit-iOS.git">https://github.com/AgoraIO-Community/VideoUIKit-iOS.git</a></p><p>When asked for the version, insert 4.0.0-preview, and select &quot;Up to Next Major&quot;. This should install at least version 4.0.0-preview.8, which is released at the time of writing.</p><p>When in doubt, you can always select “Exact” and insert 4.0.0-preview.8.</p><p>Once that package installed, the camera and microphone usage descriptions need to be added. To see how to do that, check out Apple’s documentation here:</p><p><a href="https://developer.apple.com/documentation/avfoundation/cameras_and_media_capture/requesting_authorization_for_media_capture_on_ios#2962313">Apple Developer Documentation</a></p><h4>Video Call UI</h4><p>I won’t go into much detail about the UI here, because we’re mainly focusing on the Voicemod extension, but on a high level:</p><ul><li>The ContentView contains two views: a VideoCallView (defined in the project, and a button to join the channel.</li><li>ContentView holds the active AgoraViewer, which is brought in from Agora UIKit.</li><li>When the button is pressed, the Agora UIKit method join(channel:,with:,as:) is called, which joins the Agora video channel.</li></ul><p>This is what the ContentView looks like:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8fd2c6da8aae003930444f03a9a61451/href">https://medium.com/media/8fd2c6da8aae003930444f03a9a61451/href</a></iframe><p>And the VideoCallView, without any of the voicemod additions looks like this:</p><pre>struct VideoCallView: View {<br>    @Binding var joinedChannel: Bool<br>    var body: some View {<br>        ZStack {<br>            ContentView.agview<br>        }<br>    }<br>}</pre><p>We now have a working video call app with SwiftUI and Agora UIKit. The next step is to integrate the Voicemod extension!</p><h3>Integrating Voicemod</h3><h4>Voicemod Credentials</h4><p>If you have an account with Agora and are currently signed in, follow <a href="https://console.agora.io/marketplace/extension/introduce?serviceName=voicemod">this link</a> to activate the Voicemod extension for your account.</p><p>First, you will need to activate Voicemod for your account by clicking “Activate” here:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mGuVuFaXchlrnMw-tXZvvQ.jpeg" /><figcaption>Activate extension</figcaption></figure><p>Next, enable Voicemod for the project, the same project as for the Agora App ID used in the app:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w88URs7XMrNW1q0nTLNEpw.jpeg" /><figcaption>Enable extension for project</figcaption></figure><p>Once that’s done, you can grab your Voicemod API Key and API Secret by clicking “view” below the credentials tab:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yds6uYbywRbg7youwnZOgA.jpeg" /><figcaption>Agora extension credentials</figcaption></figure><h4>Add to Xcode</h4><p>We need to add the Voicemod extension, which can also be installed via the Swift Package Manager with the following URL:</p><p><a href="https://github.com/AgoraIO-Community/Extension-Voicemod-iOS.git">GitHub - AgoraIO-Community/Extension-Voicemod-iOS</a></p><blockquote>The latest release at the time of writing is 0.1.2.</blockquote><p>This package doesn’t need importing into your Swift code. It just needs to be bundled alongside the Agora SDK in your app target.</p><h4>Enable the Extension</h4><p>When using Agora extensions, you need to enable them before joining a channel.</p><p>To do this with Agora UIKit, you can call AgoraVideoViewer.enableExtension().</p><p>This method needs to be passed the vendor (“Voicemod”), the vendor extension (“VoicemodExtension”) in this case:</p><pre>ContentView.agview.viewer.enableExtension(<br>  withVendor: &quot;Voicemod&quot;, extension: &quot;VoicemodExtension&quot;, enabled: true<br>)</pre><p>The above code snippet was added to the beginning of the joinChannel method defined earlier.</p><h4>Choose the Voice</h4><p>After joining the channel, we’ll add a small menu to the top of our view. This menu will contain all the available voices from Voicemod. On selecting the voice, we’ll call a method to make sure we’re using that voice. On selecting an option, we update a state variable, selection.</p><p>The menu is created like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/97df3faac5d2c1e0166688ba2a73ede9/href">https://medium.com/media/97df3faac5d2c1e0166688ba2a73ede9/href</a></iframe><p>The names array is defined like this:</p><pre>let names = [<br>    &quot;baby&quot;,<br>    &quot;cave&quot;,<br>    &quot;lost-soul&quot;,<br>    &quot;robot&quot;,<br>    &quot;titan&quot;<br>]</pre><p>This is what the menu looks like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/888/1*4tFJDUKO9fLnCh_Xc3yZkg.gif" /><figcaption>voicemod voice menu</figcaption></figure><p>Almost there now! The only thing left to do is define the setVoicemodParam method.</p><p>This method needs to do two things:</p><ol><li>Initialise Voicemod, passing the API Key and API Secret values.</li><li>Set the voice property so that it knows which voice effect you want to have.</li></ol><p>First, let’s define a function registerVoicemod for setting the key and secret:</p><pre>func registerVoicemod() {<br>  // Set API Credentials<br>  let dataDict = [<br>    &quot;apiKey&quot;: AppKeys.voicemodApiKey,<br>    &quot;apiSecret&quot;: AppKeys.voicemodApiSecret<br>  ]<br>  guard let setPropertyResp = ContentView.agview.viewer.setExtensionProperty(<br>    &quot;Voicemod&quot;, extension: &quot;VoicemodExtension&quot;,<br>    key: &quot;vcmd_user_data&quot;, codable: dataDict<br>  ), setPropertyResp == 0 else {<br>    print(&quot;Could not set extension property&quot;)<br>    return<br>  }<br><br>  self.voicemodRegistered = true<br>}</pre><p>Here we are using the method setExtensionProperty with the vcmd_user_data key. The API Key and API Secret properties are stored in a Swift dictionary of type [String: String]. This method from Agora UIKit allows us to pass a Swift dictionary, which is then encoded for us and passed to Agora&#39;s engine.</p><p>If something goes wrong, either setPropertyResp will be null or the response will be less than zero. But if all the steps are followed, no issues should arise!</p><p>Now the method setVoicemodParam can be defined. This method must call registerVoicemod (first time only), and then set the voice value (robot, baby etc.):</p><pre>func setVoicemodParam(to voiceName: String) {<br>  if !self.voicemodRegistered {<br>    self.registerVoicemod()<br>  }<br><br>  // SET VOICE<br>  ContentView.agview.viewer.setExtensionProperty(<br>    &quot;Voicemod&quot;, extension: &quot;VoicemodExtension&quot;,<br>    key: &quot;vcmd_voice&quot;, strValue: voiceName<br>  )<br>}</pre><h4>Conclusion</h4><p>You now have a video call app complete with the new extension from Voicemod!</p><p>There are other extensions I’d encourage you to try out, and they can all be found here:</p><p><a href="https://console.agora.io/marketplace">Dashboard</a></p><p>The same settings can be applied to an application made directly with the Agora SDK, but you will need to make sure you’re using the 4.x version of the SDK. Details can be found here:</p><p><a href="https://docs.agora.io/en/video-call-4.x-preview/landing-page?platform=iOS">Video Call v4.0.0 Preview</a></p><p>The methods, such as setExtensionProperty, used in this post are adaptations of the SDK&#39;s built-in methods relating to extension. They can be found in this file from Agora Video UIKit:</p><p><a href="https://github.com/AgoraIO-Community/iOS-UIKit/blob/4.0.0-preview.8/Sources/Agora-UIKit/AgoraVideoViewer%2BAgoraExtensions.swift">iOS-UIKit/AgoraVideoViewer+AgoraExtensions.swift at 4.0.0-preview.8 · AgoraIO-Community/iOS-UIKit</a></p><h4>Testing</h4><p>You can try out this app by following the GitHub link:</p><p><a href="https://github.com/AgoraIO-Community/Extension-Voicemod-SwiftUI">GitHub - AgoraIO-Community/Extension-Voicemod-SwiftUI: An example of using the Voicemod Voice Changer with SwiftUI</a></p><h4>Other Resources</h4><p>For more information about building applications using Agora SDKs, take a look at the <a href="https://docs.agora.io/en/Video/start_call_ios?platform=iOS&amp;utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=extension-voicemod-swiftui">Agora Video Call Quickstart Guide</a> and <a href="https://docs.agora.io/en/Video/API%20Reference/oc/docs/headers/Agora-Objective-C-API-Overview.html?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=extension-voicemod-swiftui">Agora API Reference</a>.</p><p>I also invite you to <a href="https://www.agora.io/en/join-slack/">join the Agora Developer Slack community</a> to meet with our developer team as well as other like-minded developers and technical enthusiasts.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=faefec5a564" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Agora Releases Native SDK v3.5.1]]></title>
            <link>https://maxxfrazer.medium.com/agora-releases-native-sdk-v3-5-1-64ac1e4ce752?source=rss-2f5df90bc0eb------2</link>
            <guid isPermaLink="false">https://medium.com/p/64ac1e4ce752</guid>
            <category><![CDATA[video-conferencing]]></category>
            <category><![CDATA[webrtc]]></category>
            <category><![CDATA[real-time-communication]]></category>
            <category><![CDATA[agora]]></category>
            <dc:creator><![CDATA[Max Cobb]]></dc:creator>
            <pubDate>Mon, 18 Oct 2021 09:26:37 GMT</pubDate>
            <atom:updated>2021-10-18T09:26:37.997Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>Voice and Video SDK v3.5.1 now available for on all supported native mobile and desktop platforms. See what’s new below!</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rzJAUhhGRY8sejgiacaSPw.png" /></figure><blockquote>Agora Native SDK 3.5.1 was released on 14 October 2021.</blockquote><h3>New Features ✨</h3><h4>Super Resolution (beta) 🪟</h4><p>Effectively boost the resolution of a remote user’s video seen by the local user.</p><p>You can call <a href="https://docs.agora.io/en/Video/API%20Reference/oc/Classes/AgoraRtcEngineKit.html#//api/name/enableRemoteSuperResolution:enabled:">enableRemoteSuperResolution</a> to enable super resolution, specifying the remote user&#39;s uid. Make sure the AgoraSuperResolutionExtension framework is also included in your project.</p><h4>Background Blurring 😶‍🌫️</h4><p>Virtual backgrounds was added to the SDK since 3.5.0, and now to further enrich that, there is an option to apply a blur to the background. Set the type of the custom background image to AgoraVirtualBackgroundBlur, apply a blur_degree, and the background will be blurred.</p><h4>Device Flash 📸</h4><p>isCameraTorchSupported and setCameraTorchOn have been added to the C++ APIs for both iOS and Android.</p><h4>Media Stream Relay Pausing/Resuming ⏯</h4><p>Methods have been added to quickly pause and resume cross-channel media stream relays; as well as additional delegate methods to check if the pause and resume commands were successful.</p><h4>Pushing External Audio to a Specified Position 🎤</h4><p>pushExternalAudioFrameRawData and pushExternalAudioFrameSampleBuffer have been updated to have an additional parameter. This parameter, sourcePos, allows you to push the external audio frames to one of three positions: after audio capture, before audio encoding, or before local playback.</p><p>Additionally, setExternalAudioSourceVolume has also been added; allowing you to set the volume of external audio frames at a specified position.</p><h4>Music and Audio File Updates 🎶</h4><p>Playback speed, audio track, and channel mode of a music file can be set in code.</p><p>This update deprecates getAudioMixingDuration method in favour of getAudioFileInfo. getAudioFileInfo still gets information such as duration, but returns it in an AgoraRtcAudioFileInfo object in the didRequestAudioFileInfo delegate method, found <a href="https://docs.agora.io/en/Video/API%20Reference/oc/Protocols/AgoraRtcEngineDelegate.html#//api/name/rtcEngine:didRequestAudioFileInfo:error:">here</a>.</p><h3>Improvements</h3><h4>5G Quality Testing ⚡️</h4><p>When monitoring the connection quality of a device, a new property has been added to the AgoraNetworkType enum, AgoraNetworkTypeMobile5G(6), or .mobile5G with Swift, and NETWORK_TYPE_MOBILE_5G on Android.</p><p>These are just a handful of the new features to the SDK in v3.5.0. To see the full release notes for the native platform you’re using, check out one of the following links: <a href="https://docs.agora.io/en/Video/release_ios_video?platform=iOS#v351">iOS</a>, <a href="https://docs.agora.io/en/Video/release_android_video?platform=Android#v351">Android</a>, <a href="https://docs.agora.io/en/Video/release_mac_video?platform=macOS#v351">macOS</a>, and <a href="https://docs.agora.io/en/Video/release_windows_video?platform=Windows#v351">Windows</a>. Head to Agora’s <a href="https://docs.agora.io/en/Video/downloads?platform=All%20Platforms">downloads page</a> to get the latest SDK now.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=64ac1e4ce752" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>