ESO’s Performance Improvements Plan (Yeah right…)

We’re working hard to make sure The Elder Scrolls Online runs as best as it possibly can. Read our plan for the game’s upcoming performance improvements from ESO’s Creative Director Rich Lambert and check back monthly for status updates.

January 2020 Update

In January, the team continued work on everything coming for Update 25, including improvements to frame rate and load times. You can test many of these improvements yourself on the Public Test Server for PC/Mac right now! Don’t forget, when Update 25 launches, you will be required to re-download the game client. Look out for more information on this process in the coming weeks.

Q4 2019: Update 24

  • Memory management overhaul and behind-the-scenes combat ability improvements phase 1
    • This is now live on all servers (PC/Mac, Xbox One, and PlayStation®4) and stability is greatly improved on all platforms.
    • Additional memory-related fixes are coming for PS4™ and Xbox One in an incremental patch on February 5.
    • Fixed for a few remaining bugs coming with Update 25.
  • Complete rewrite of the Looking for Group system
    • The initial system rewrite work is now live on the PC/Mac, PlayStation®4, and Xbox One servers, including fixes for some edge-cases with pre-formed groups.
    • The Undaunted Celebration Event was a success! Thank you to everyone who participated and helped us run all the latest Looking for Group system fixes through their paces.
    • The Dev team is now focused on investigating some edge cases (Battlegrounds and Dungeons) and cleaning up the last few remaining bugs. Look for more fixes with Update 25.

Q1 2020: Update 25

  • Patching overhaul
    • Work is complete and successfully launched on the ESO Public Test Server. Everything is on track to release with Update 25.
    • Note: This will result in approximately 16GB of savings on disk for PC/Mac, and approximately 30GB savings on PlayStation®4 and Xbox One.
    • Load times have been significantly reduced
    • Reminder: Every ESO player, on every platform, will have to re-download the game client when this change happens, probably with the launch of Update 25.
  • Faster loading fixtures and characters
    • Work is complete and successfully launched on the ESO Public Test Server. Everything is on track to release with Update 25.
    • As noted above, load times have been significantly reduced.
  • Behind-the-scenes combat ability improvements phase 2
    • Work is complete and successfully launched on the ESO Public Test Server. Everything is on track to release with Update 25.
    • We are working on investigating and fixing a number of bugs identified on the ESO Public Test Server – specifically with Block. Work is in progress on resolving them in future PTS incremental patches.
  • Frame rate improvements part 1
    • Effects Manager: Initial effects manager work and bug fixing is complete, on the ESO Public Test Server, and on track to release with Update 25. During testing, we identified additional items we need to address in the effects manager. These will be delivered with Update 26. See the Update 26 section below.
    • Multi-threading Work: This work is complete and successfully launched on the ESO Public Test Server. This is on track to release with Update 25.
    • Frame Rate Improvements: Initial testing on the ESO Public Test Server shows reduced frame rate hitching and spikes – frame rate is more stable as a result and the game feels smoother and more responsive in combat. We will be monitoring this closely, especially in highly populated and high combat situations.

Q2 2020: Update 26

  • Server optimization & stability
    • Pets: We are going to rewrite how pets are handled to be more performant (work better, more efficiently, and take up less overhead/resources on the server.) Initial code work is complete and in internal bug fixing. This is on track to release with Update 26.
    • Player Character Loading: We will make the player character loading process multi-threaded on the server to improve performance. This work is ongoing and on track to release with Update 26. Initial code work is complete and in internal bug fixing. This is on track to release with Update 26.
  • Frame rate improvements phase 2
    • Multi-threaded Fixture Creation on the Client: This work centers around how art assets are constructed and drawn on the client, ultimately spreading this work out over multiple cores. This work is on track to release with Update 26.
    • Fixture Creation: Initial code work is complete and in internal bug fixing. This is on track to release with Update 26.
    • *New* Character Update: This work centers around how characters are drawn on the client, spreading work out over multiple cores. Initial code work is in progress and on track to release with Update 26.
    • *New* FX Update: This is additional work that we identified as during the first part of the effects manager improvements that are coming with Update 25. Initial code work is near complete and in internal bug fixing. This is on track to release with Update 26.
  • Behind the scenes combat ability improvements phase 3
    • AoE Ability Performance on the Server: This work focuses on making these ability types more specialized so that they are more efficient on server performance. Note: The overall player experience for these abilities (damage/functionality) should not change. This work is in progress and on track to release with Update 26.

Note: In early Q1 2020, we will start adding Q3: Update 27 and Q4: Update 28 feature work to this plan.

December 2019 Update

This month, we continued to roll out incremental patches to help address stability and issues with the Activity Finder. Work for Update 25 continues with most fixes on track, including the patching overhaul that shall require ALL ESO players re-download the game client (as a gentle reminder).

Q4 2019: Update 24

  • Memory management overhaul and behind-the-scenes combat ability improvements phase 1
    • This is now live on all servers (PC/Mac, Xbox One, and PlayStation®4.)
    • Incremental patches with additional stability fixes and improvements also went out on all platforms. Stability is greatly improved on all platforms. We have one more PS4™ incremental patch planned for January 2020 to address some remaining PS4™-specific stability issues.
  • Complete rewrite of the Looking for Group system
    • The initial system rewrite work is now live on the PC/Mac, PlayStation®4, and Xbox One servers.
    • Some additional work is ongoing to address a few outstanding issues (edge-cases for pre-formed groups) that will be fixed by the time of this writing. Look for an Undaunted test early next year to stress test the fixes.

Q1 2020: Update 25

  • Patching overhaul
    • Work is complete and in final internal testing/bug fixing, and on track to release with Update 25.
    • Reminder: Every ESO player, on every platform, will have to re-download the game client when this change happens, probably with the launch of Update 25.
  • Faster loading fixtures and characters
    • Work is complete and in final internal testing/bug fixing, and on track to release with Update 25.
  • Behind-the-scenes combat ability improvements phase 2
    • Work is complete and in final internal testing/bug fixing, and on track to release with Update 25.
  • Frame rate improvements part 1
    • Effects manager work and bug fixing is complete. This is on track to release with Update 25.
    • Multi-threading work is complete and in final internal testing/bug fixing. This work will allow more game systems (specifically the fixture update process and renderer for this round) to utilize multiple cores, improving overall performance on the game client. This is on track to release with Update 25.

Q2 2020: Update 26

  • Server optimization & stability
    • Pets – We are going to rewrite how pets are handled to be more performant (work better, more efficiently, and take up less overhead/resources on the server.) This work is ongoing and on track to release with Update 26.
    • Player character loading – We will make the player character loading process multi-threaded on the server to improve performance. This work is ongoing and on track to release with Update 26.
  • Frame rate improvements phase 2
    • Multi-threaded fixture creation on the client – This work centers around how art assets are constructed and drawn on the client, ultimately spreading this work out over multiple cores. This work is on track to release with Update 26.
  • Behind the scenes combat ability improvements phase 3
    • AoE ability performance on the server – This work focuses on making these ability types more specialized so that they are more efficient on server performance. Note: The overall player experience for these abilities (damage/functionality) should not change. This work is on track to release with Update 26.

Note: In early Q1 2020, we will start adding Q3: Update 27 and Q4: Update 28 feature work to this plan.

November 2019 Update

This month featured a lot of additional work on the Looking for Group system/Activity Finder tool. For more information concerning the recent issues with the Activity Finder tool and our plan to tackle them, please see this forum post.

Q4 2019: Update 24

  • Memory management overhaul and behind-the-scenes combat ability improvements phase 1
    • This is now live on the PC/Mac servers and coming to the PlayStation®4 and Xbox One on November 5 with Update 24.
    • We are monitoring for live feedback and fixing bugs. The next incremental patch on PC/Mac and XB1/PS4™ will have additional stability fixes and improvements.
  • Complete rewrite of the Looking for Group system
    • This is now live on the PC/Mac, PlayStation®4, and Xbox One servers.
    • Since this went live, we’ve made some adjustments and fixed some performance-related problems on the live servers including some last week. We are still working on fixing bugs as we identify them. We have additional work in progress to further improve the system that should be completed before Update 25 launches.

Q1 2020: Update 25

  • Patching overhaul
    • Work is ongoing and on track, with audio assets and the patching back-end in final stages. Everything else is complete and in testing.
    • Reminder: Every ESO player, on every platform, will have to re-download the game client when this change happens, probably with the launch of Update 25
  • Faster loading fixtures and characters
    • Work is complete and in internal testing.
  • Behind-the-scenes combat ability improvements phase 2
    • Work is ongoing and on track – no new major updates at this time.
  • Frame rate improvements part 1
    • Initial work on the effects manager is complete and in internal testing. This is on track to release with Update 25.
    • Initial work on multi-threading is complete and in internal testing. This work will allow more game systems (specifically the fixture update process and renderer for this round) to utilize multiple cores, improving overall performance on the game client. This is on track to release with Update 25.

Q2 2020: Update 26

  • Account Database “Cold Storage”
    • This work is not yet started. Transfer system work (noted above) and additional Looking for Group system work needs to be completed before we start this. As such, this has been shifted out to Q3 2020 (Update 27).
  • Server optimization & stability
    • Pets – We are going to rewrite how pets are handled to be more performant (work better, more efficiently, and take up less overhead/resources on the server.) This work is ongoing and on track to release with Update 26.
  • Player character loading – We will make the player character loading process multi-threaded on the server to improve performance. This work is ongoing and on track to release with Update 26.
  • Frame rate improvements phase 2
    • Multi-threaded fixture creation on the client – This work centers around how art assets are constructed and drawn on the client, ultimately spreading this work out over multiple cores. This work is on track to release with Update 26.
  • Behind the scenes combat ability improvements phase 3
    • AoE ability performance on the server – This work focuses on making these ability types more specialized so that they are more efficient on server performance. Note: The overall player experience for these abilities (damage/functionality) should not change.

October 2019 Update

Here’s the October update for our ongoing performance optimization work. As many of you know, we’ve already launched Update 24, and subsequently had to make some adjustments to the new Looking for Group system.

Q4 2019: Update 24

  • Memory management overhaul and behind-the-scenes combat ability improvements phase 1
    • This is now live on the PC/Mac servers and coming to the PlayStation®4 and Xbox One on November 5 with Update 24.
    • We are monitoring for live feedback and bugs and will assess if more work is needed.
  • Complete rewrite of the Looking for Group system
    • This is now live on the PC/Mac servers and coming to the PlayStation®4 and Xbox One on November 5 with Update 24.
    • We’ve already had to make some adjustments to this on the live servers in the week after Update 24 went live to fix some performance-related problems. It is now functioning well and is holding up to the player load, but we have already found a few ways we can make the system better and will be doing additional work that should be completed before Update 25 launches.

Q1 2020: Update 25

  • Patching overhaul
    • Work is ongoing and on track with internal bug fixing and asset clean-up as the focus.
  • Faster loading fixtures and characters
    • Work is ongoing and on track – no new major updates at this time.
  • Behind-the-scenes combat ability improvements phase 2
    • Work is ongoing and on track – no new major updates at this time.
  • Frame rate improvements part 1
    • Effects manager work is in progress and on track.
    • We have identified some multi-threading work that is needed, and that is now in progress. This work will allow more game systems (specifically the fixture update process and renderer for this round) to utilize multiple cores, improving overall performance on the game client.
  • Server optimization & stability
    • This work has moved out to Update 26 (see below)

Q2 2020: Update 26

  • Account Database “Cold Storage”
    • This work is not yet started. Transfer system work (noted above) needs to be completed before we start this. We still believe this work will be completed in time for Update 26.
  • Server optimization & stability
    • Pets – We are going to rewrite how pets are handled to be more performant (work better, more efficiently, and take up less overhead/resources on the server.)
    • Player character loading – We will make the player character loading process multi-threaded on the server to improve performance
  • Frame rate improvements phase 2
    • Work is ongoing and on track – no new major updates at this time.
  • Additional bug fixes and other updates
    • Work is ongoing and on track – no new major updates at this time.

September 2019 Update

Below is a summary for September to give you an update on how work is coming along for Updates 24 and 25, and a look ahead to Update 26. In short, all work is still on track to roll out with the corresponding game update. Please see below for specific details.

Q4 2019: Update 24

  • Memory management overhaul
    • The major work for this game code is complete, and it is now in public testing on the ESO PTS. We’re fixing bugs as they arise. This is on track for release with Update 24.
    • Through internal testing, the team uncovered a few additional smaller memory-related subsystems that we can also optimize/improve. Work for that has begun and will be completed for Update 25.
  • Behind-the-scenes combat ability improvements phase 1
    • Work for this is complete and now testable on the ESO PTS. This includes the sprint and mount sprint performance updates and Champion passive performance updates mentioned in our August game performance plan update. This is on track for release with Update 24.
  • Complete rewrite of the Looking for Group system
    • This work is complete, and it is now in public testing on the PTS. We’re fixing bugs as they arise. This includes both the dungeons and Battlegrounds portions of the game code that we mentioned in our August game performance plan update. This is on track for release with Update 24.

Q1 2020: Update 25

  • Patching overhaul
    • The game code portion of this work is complete and is being prepped for internal testing. Note: This is going to require a significant amount of testing, but we are still on track for an Update 25 delivery.
  • Faster loading fixtures and characters
    • This is tied closely to the patching overhaul above. The game code portion of this work is complete, and assets are being processed and prepped for internal testing. This is on track for release with Update 25. Note: Like with the patching overhaul, these changes also require significant testing.
  • Behind-the-scenes combat ability improvements phase 2
    • Work is ongoing and on track – no new major updates at this time.
  • Frame rate improvements part 1
    • The initial investigating portion of this has concluded, and we have identified the areas in which we need to focus our efforts. Our #1 priority for this work is the effects manager. This system drives everything related to abilities in the game including but not limited to particle effects, animations, and movement properties.
    • We are also evaluating other systems for CPU performance improvement opportunities including but not limited to fixtures and characters.
  • Server optimization & stability
    • Investigation is still ongoing and work is on track.

August 2019 Update

Below is a summary for August to give you an update on how work is coming along for the next couple updates. In short, all work is on track to roll out with the corresponding game update. See below for specific details.

Q4 2019: Update 24

  • Memory management overhaul
    • The game code for this is complete and we’re now in the internal testing phase. This is on track to release with Update 24.
  • Behind-the-scenes combat ability improvements phase 1
    • Performance updates on the most consumptive combat-related abilities have been completed and are now in internal testing.
    • Champion passive performance updates are also complete and in internal testing. Note: Champion System work for this update is solely focused on game performance. We are still in the design phase for what we want to do in the future with the overall system.
  • Complete rewrite of the Looking for Group system
    • The dungeons portion of the game code is complete and in internal testing. (We completed the first scale test during the Update 23 PTS, week 2.)
    • The Battlegrounds portion of the game code is being finalized and will be in internal testing soon. This is on track for an Update 24 delivery.

Q1 2020: Update 25

  • Patching overhaul
    • Work continues on identifying duplicate/unused data. This is on track to release with Update 25.
  • Faster loading fixtures and characters
    • The game code for this is complete and we’re now in the internal testing phase. Note: These changes do require significant testing, but we are still on track for an Update 25 delivery.
  • Behind-the-scenes combat ability improvements phase 2
    • No new updates on this one at this time. Work is still ongoing and on track.
  • Frame rate improvements part 1
    • Work is scheduled to begin after the Update 24 memory management overhaul polish/bug fixing is complete.
  • Server optimization & stability
    • Investigation is still in progress, with the current focus on combat-related game code

***

With the launch of Elsweyr, we wanted to give you an update on our ongoing performance optimization initiatives, our future plans, and when–generally–you can expect to see the improvements. This will be a living document, and it will be updated as new information becomes available.

We know that ESO performance has degraded for some players, especially on the base PS4™ and XB1. We’re not happy with this, and we know how important it is for us to alleviate the situation and give regular updates on what we are doing. A game that’s still going strong after five years brings a number of challenges to keep it running as smoothly as possible. Technology and player behavior changes over time, and of course the amount of available content continually expands. All of these can impact performance, and they have in ESO.

This article is the first step in a new process of making sure that everyone knows what we’re working on. The goal is to outline the things we have planned, explain why a particular item is important to overall stability or performance, and provide a target for when we think the work is going to be complete and on the live servers. Please keep in mind that some of these fixes reach very deep into the architecture of the game and require a lot of care, time, and testing. Because of this, the target dates on some of the initiatives may shift as work progresses.

Q4 2019: Update 24

  • Memory management overhaul
    • Fundamental changes to how we handle memory use in ESO to reduce memory fragmentation
    • User Experience: Greatly improved stability (i.e., less crashing / dashboarding), especially on base consoles, as well as potential minor performance improvement
  • Behind-the-scenes combat ability improvements phase 1
    • This is the first of an ongoing initiative to rewrite some of the more consumptive combat abilities to lessen server load
    • User Experience: This is the first of many behind-the-scenes combat optimizations and may not have a huge impact, but over time this will lessen the server load and allow smoother and more responsive combat with larger groups
  •  Complete re-write of the Looking for Group system
    • Ground-up rewrite of how the system handles queues of people in both dungeons and battlegrounds—focused on reliability and speed, especially under load
    • User Experience: The LFG systems work as expected, even under heavy load

Q1 2020: Update 25

  •  Patching overhaul
    • Big changes to the way ESO is downloaded, patched, and stored on local hard drives. We need to refresh ESO’s patching and file manifest data to remove duplicate files and older data that is no longer needed. This will result in a major reduction of space taken up by the client on the player’s local hard drive. Console players will see a significantly larger impact compared to PC, because of the way those patching systems work
    • User Experience: Major improvement in load times, streaming of assets, and reliability of asset loads, especially on consoles
    • NOTE: Every ESO player, on every platform, will have to re-download the game client when this change happens, probably with the launch of Update 25
    • NOTE: After this change happens, patch size going forward should also be reduced
  • Faster-loading art fixtures and characters
    • As part of the client changes above, we will be moving to faster/newer/better compression libraries for art fixtures and character loading—these are two of the most “expensive” loading tasks
    • User Experience: Better load times, fewer “shadow” player silhouettes
  • Behind-the-scenes combat ability improvements phase 2
    • The next round of behind-the-scenes ability streamlining and optimizations
    • User Experience: A smoother, more responsive experience in combat situations
  • Frame rate improvements phase 1
    • We have time built in the schedule here for general frame rate improvements; more details on this in future updates
    • User Experience: Smoother, faster frame rates
  • Server optimization & stability
    • Server optimizations are being investigated and assessed, with the focus on CPU and Bandwidth usage on the server—especially on the combat handler and messaging
    • User Experience: A more responsive experience, especially in intense or high bandwidth situations (i.e., places where large groups of players are found)

Q2 2020: Update 26

  • Account Database “Cold Storage”
    • There are millions of accounts in our database. The goal of this task is to split the character database into “active” and “inactive” shards to remove the need to pool all the character records for all logins
    • User Experience: Faster account login and character loads
    • NOTE: Players logging in for the first time after being inactive for a few years will notice an initial longer load while their account is transferred from inactive to active
  • Frame rate improvements phase 2
    • The team has time built into Update 26 to identify more areas for frame rate improvements. Please note that this takes place after the frame rate improvements in Update 25, so this will piggy-back on those
    • User Experience: Better, smoother frame rate
  • Additional bug fixes and other updates
    • The NPC and monster “pathing” system will be overhauled to result in fewer movement bugs
    • NOTE: We are always working on identifying outstanding bugs and fixing them. If any rise to the level of needing to be included in this regularly updated post, we will do so here

As you can see, we have a huge amount of work ahead of us. There’s a ton of stuff to do and as time progresses, you should expect that some of these items may change—especially the investigative tasks since we aren’t 100% sure what will be uncovered. Regardless, we will keep you updated. Watch this article and our official ESO social channels for updates at least once a month.

Thanks, everyone, for being patient with us while we work through these issues. See you all in Tamriel!

—Rich

***

That’s it! Don’t forget to keep an eye on this article for monthly updates on the status of the above improvements—we can’t wait to bring all these updates to you.

21 total views, 1 views today