This webinar covers:
• How to prepare and plan for migration
• Stories from people with first-hand experience
• Approaches, options and tools for migration
Irene He, Atlassian Certified Master and Manager of Atlassian Solutions discuss DragonAgile’s approach to cloud migration and share tips on cloud migration and consolidation. Our guest speaker Pawel Rozek is a vetted Jira Administrator at McKesson and will discuss his own journey with cloud migration.
Irene: Thank you very much for joining us and I hope everyone is doing well and please feel free send in your questions! Submit your questions in the chat window while we’re doing the
presentation. We will address them at the end of our webinar.
Irene: I can see people joining from all over the places, some people even from Europe, so I’m glad you don’t have to fly into Canada because we are snowing here. So shall we get started? Let’s get started. Should we introduce ourselves first? Do you want to start first?
Pawel: Sure, so my name is Pawel Rozak. I’m a vetted Jira administrator. I currently work for McKesson Canada and I’m a part-time Jira administrator and I also am in DevOps.
Irene: Hi everyone, I’m Irene and I’m working for DragonAgile and I actually begin the DragonAgile office! Globally, we are an Atlassian Platinum Solution Partner so myself is a Atlassian Certified Master.
Irene: So just a little bit about DragonAgile. Our main office is located in Waterloo and we offer licensing with added value, implementation, customization, training, and support for Atlassian products.
Irene: Here is today’s agenda. First up, Pawel will tell his migration story and I will interview him with a few questions. After that, I will tell my migration story while Pawel do interview for me. Next, we will take questions from the audience and I hope everything answer them. At the end, we will wrap up the webinar. Shall we take off now?
Irene: So Pawel, tell us about your recent cloud migration experience.
Pawel: Sure, Irene. First, I just wanted to say thanks for having me. Recently I’ve done some Jira Cloud migrations. My management team came to me and asked if I could help with the Jira Cloud migrations. Basically, we had three different Jira Cloud instances. One of them was being decommissioned and all the other projects were required to move to our new Jira and the target instance were being migrated to a different one by another team. They asked me to move a couple of projects from that instance into our target cloud instance. And then my manager came a couple days later and asked if you can do a few more projects from a different Jira cloud instance and migrate them into the target store as I’m doing everything all at once. So that’s basically my request that I had and I move forward with it.
Irene: Sounds great! So with this requirement, how in the world did you start with your journey?
Pawel: Before the migration, you have a few things you need to consider. First of all, you need to know if your Jira cloud instance has any NextGen projects. That’s really important because when you have NextGen projects, what’s happening is you need to actually migrate them to the old-style projects. The NextGen projects currently are not migratable to JIRA server and because of the way we perform the migration, we have to either delete the ones that we didn’t need or migrate them to regular projects.
Pawel: I also had to figure out if I had any plugin data that are required to be migrated. That’s also really important to know ahead of time because if you do have any plug-in data, we need to consider some steps and look at the documentation for the vendor to see if they actually can migrate.
Pawel: You need to check the differences between priorities and instances. For example, one of our instances just had priorities as P1, P2, P3 and the other one had priorities High, Highest, Normal, Low, Lowest, and TBD. I needed to merge those, get an agreement among all these teams and from all the instances to make sure that they’re okay with us changing the priorities.
Pawel: Also, a very important thing was fields. Sometimes one team calls for example a scrum team field as a text field and the other one is a drop-down. So we had to creatively adjust those and consider changing those.
Pawel: Duplicate resolutions was another thing that was also considered because in one instance, we have cancelled with two L’s and one instance we have canceled with one L so we have to consider changing to one way or the other.
Pawel: The last thing to consider specifically for me was do I know to set up a Jira server instance to perform these merges. This is important – if you don’t know how to do that, then you have to look at other options.
Irene: Sounds like a lot to consider for your migration! With all this pre-screening, analyzing, what were the migration options that you considered?
Pawel: Migration options that I considered was performing using Jira Server. As you can probably tell, that’s the way we’ve chose to perform it. There’s also looking at CSV exports and imports. Basically, you export all your data using a CSV and you import it back into the target instance. And the last thing we considered was Jira Cloud Migration Assistant Early Access Program, which I have a link there. Basically, this is Atlassian doing it for you but we’ve decided to perform it ourselves and using Jira server method that you can see there on the screen.
Pawel: We downloaded the backups from Jira Cloud, created three stores into server and then uploaded everything back into the target instance of Jira cloud.
Irene: Hmm, so I think that you’ve said that the first option to merge the instance on server site first before you migrate to cloud looks like a good approach for your scenario. But then how did you prepare before merging the instances?
Pawel: So we’ve done some prep work that that was required. We deleted a bunch of unused workflow schemes, unused workflows, issue types, screen schemes. We deleted unused screen scheme, screens, issue type schemes, and issue types except for the built-in ones that come with JIRA.
Pawel: This is probably one of my learnings as well. I also made a mistake and initially, during my testing I deleted JIRA Service Desk issue types and then I was looking at restoring that.
Pawel: I also removed unused security schemes, notifications, permissions, field configurations and schemes and additionally fields too. Oftentimes what happens is fields are just no longer used and no longer required, so we just clean those up. One of the things was also field contexts. We redeemed those based on the instance that they were coming from as a preparation so that it’s easier to understand where it’s coming from, which projects are using it. And we also cleaned up some groups.
Irene: Wow, that looks like lots of clean up and manual work for you! You must be very busy at that time. So when the time comes to merge, did you need to lock down the instance and how did you do it? Did that affect your team’s work?
Pawel: Sure, so we did establish a blackout period where no requests were being processed by Jira administrators and we advised the user base that we need about two weeks, one to two weeks of prep work for all those things that I mentioned earlier to happen. All the field clean up and all that other stuff so we established. The users were still able to queue up their requests – it just wasn’t done until after the migration was done. The other thing we’ve done was we removed the administrative privileges from users that should not have and actually pretty much everybody except for myself and one more user, just in case you know we required additional work.
Pawel: We established a list of fields, screens, workflows and necessary information for the migration and we provided that to the user to say, “This is what you’re going to have to live with for the next two weeks until the production migration happens.”
Irene: Again, sounds like a lot to prepare and then I think this is really thoughtful. With all clean-up and preparation, did you need to test out first on your trial instance before your final production migration?
Pawel: Great question, Irene. So yes, we had to set up a cloud instance of Jira to test it out so that we don’t disrupt production stuff. But basically after all the merging happened in the Jira server, we uploaded to this test cloud. Atlassian allows you to set up a cloud instance, it gives you seven days. But you can also request an extended trial from Atlassian that I think is up to 30 days from my understanding and that allows you to do all your testing.
Pawel: I’ve also documented a procedure for reference for production on day reference because when you’re performing production, you’ve done it so many times you’re prone to make mistakes and you’re prone to forget something that is important and you may possibly forget that during production and then you have to do it all over! But following your script is great because you’re just doing it as if you’re doing the testing and everything should and usually does, everything should be perfect!
Irene: I totally agree with you that good documentation definitely helps. We all know that even we have tried it out, test out successfully once, things can still go wrong. Following good documentation with the ready tested out steps definitely will smooth out the migration. I think this is a really good point. With all this, can you describe your migration procedure?
Pawel: Sure! So I touched on it a couple slides back but basically, the procedure is documented on the Atlassian website. Documentation link is right there in the slides for anyone that’s interested. What you do is you create a cloud backup and once you have the Jira backup created, you restore it onto your server. So you create it from your source instances first. You create the backup on the JIRA server. You keep the backup in behind so that time you have it for when you’re actually performing the merger of the project. Once that that was established for all the source instances, you create the same thing for the target instance. Then you perform project migrations using Project Import Utility from Atlassian. And then what I did in between those were backups after every single project so that I don’t have to go back all the way to the beginning if something goes wrong, so backups in between. Once everything is finalized, what you do is create a backup using Jira server which you have a link for there as well, to Atlassian documentation. And then you import that backup into the Jira Cloud file.
Irene: That sounds good. So what did learn from this migration and do you have any particular issues that you run into?
Pawel: So I’ve had quite a few lessons learned. One of them was when restoring a backup from Jira Cloud to Jira server, you must have the same plugins installed on your server as you have on all of your Jira Cloud instances. It’s not well documented in Atlassian documentation, but through some communication with Atlassian, I’ve learned that Atlassian allows you for trial version of all the plugins for 30 days. If you really need more than 30 days to perform the migration, you can ask for another trial license for all the plugins so I actually recommend that you use that.
Pawel: The other thing was performing Check Integrity which is built into Jira server. Your server basically checks the integrity of the database and verifies that you have no issues. Another thing is Botron Software also offers an integrity checker which is a free tool currently and it helps you eliminate all the problems that you have in your instance, for example cap duplicate issue types. And that will pick it out and it will tell you we got to fix that.
Pawel: So another lesson that I’ve learned was check your user base. We had an incident where one of our users was an owner of one of the instances and it was a regular user of the other so they had an administrative account on one instance and on the other instance just their full name. Basically, after performing all the merges, we didn’t realize this and it was a good thing we were doing a lot of testing. After performing all the mergers and uploading into our test instance of JIRA cloud, we discovered that nobody had access to the instance except for myself because I was the one that owned it. After reaching out to Atlassian through support tickets, they were able to quickly discover that it was that user. What happened was the external ID of that user was the same on both usernames, they had different usernames so it was causing issues. Atlassian provided some really good steps and support to avoid that, which I’ve also documented in my procedure.
Pawel: The other thing I’ve mentioned a little bit earlier, but you have to take a backup every single time you do something. Once you perform your migration of the project, take a backup on your Jira server because going back to the very beginning is much harder than just performing one change at a time.
Pawel: Verifying that the attachments were properly migrated during the import – I had an instance where one of the projects and attachments wouldn’t migrate so I migrated manually which was basically copy from the folder to folder but still, I had to perform that action myself rather than rely on the tool to do it for me.
Pawel: And the last thing but this is probably one of the most important things, was the database. So I have my test database sitting there and I started restoring my backups and just overriding what was in there. However, oftentimes what happens is there’s some leftover data in there that needs to be cleaned up so I recommend that that you drop your database instead of overriding it and or just recreate a new database before you start your restore from a backup as the data will not be overwritten and left over behind.
Irene: Do you have more tips and tricks you want to share?
Pawel: I have a few tips and tricks you can view on the next slide. Basically, have the users perform a check of your test instance. This is really important because as an administrator, you do things a certain way but your users do things very strangely sometimes. They have certain ways of working. And they have strange ways of doing things that you can’t think of and you’re only one person, you know hundreds of users sometimes, so have them check it and tell you if there’s any issues with your instance.
Pawel: Before performing merges, delete any projects and associated workflows so that your instance backup is a lot smaller. The smaller the backup, the faster you can perform the merges. The last thing is that when you have smaller backups, there’s less chance of you making mistakes with projects that don’t necessarily have to be migrated.
Pawel: Minimize the amount of users and groups in the instance and verify that those users should have access to the instance. Often times, especially if you’re not connected to your work network like an LDAP server, there’s often users that just leave their company in and that cleanup should happen. This is a great opportunity to perform that, as well as arranging the group so that they’re very similar across all the instances so that users can adjust.
Pawel: If merging back into the same instance that’s one of your source targets, just make sure that you download all of your plugin data before you restore in production as that is that is one of the things that you cannot get back.
Irene: I think that those are all very practical, good tips. Thank you for sharing and especially I like that you’re also taking care, taking the steps to handle app data since we know right now it’s also one painpoint to migrate data and keep the app data.
Pawel: So Irene, what’s your story? Tell me about your client’s requirements.
Irene: So I also have a migration story that I want to share with the audience. I have a customer and they have a server instance as well as a cloud instance, and both instances are really actively used by the internal team and also the external clients. Lately, they had a lot of issues with their server instance because that instance is hosted on prem on AWS, which has been hacked multiple times. Sometimes, it just causes the whole system to go down and costs a developer solution working hours. They do not have the resources to take care of this Server instance.
Pawel: Yeah, so this would be a perfect example of somebody who would want to migrate to Jira cloud because it’s all maintained by Atlassian.
Irene: Yes, that’s why they come to us. So secondly, for cost reasons; since they now have a cloud instance and most new projects will be created with this cloud instance so consolidate both instances into one, it makes much more sense for the cost reasons.
Irene: So with this goal in mind, our first of thought would be doing a full migration for a customer which means we would like to put all of the projects from the server instance all together into an existing cloud instance. That means in order to use this approach, we have to first export the cloud instance just like you described how you did the merge. So the merge can only be done on the server site so we have to export the cloud instance to server and merge with their server instance. After the merge is complete, we can export again into the cloud destination. That is our first thought.
Pawel: What was important for your client when migrating all of their data?
Irene: Yeah, that’s a good question! For my client, for lots of customers, downtime is always a very critical factor they have to think about. Also for this particular customer, app data, which applies to many customers, is also very important. For this client, they rely on the work logs to bill their clients so they cannot afford losing, for example Tempo Timesheets, that’s the app they are using so they cannot avoid to lose the work logs with Tempo Timesheets because that’s how they bill their customers. That’s very critical for them.
Pawel: So what were the pros and cons of the options they considered then?
Irene: Like you said, our first approach would be doing a full migration and consolidation but when we analyzed further, we realized that exporting Cloud to server first we would lose that important app data like time log. Even if we could merge that with the server instance together and export back into the Cloud instance then we also lose app data on the server side so we lose both sides. That doesn’t sound like the customer is not very happy with this approach.
Irene: So we discussed further with the customer and then realized that the existing Cloud instance is much more important for them and on the server instance, there are only a limited number of projects that they actually need export and consolidate and move to a Cloud instance. The rest of the project can be archived because they are old projects. With this new information, we came out with our second option so our second option would be doing a partial migration. What that means is we will still keep the cloud instance but only moving a limited number of projects from server to the cloud. This way, we keep the app data on Cloud. So they have all the log information.
Irene: Luckily, another background information is that prior to this migration, we have helped customer doing some work on that cloud site so we have created some generic project templates for this customer on the cloud so they are happy to continue using those project templates for any future projects, even for the project of moving from the server to crowd, so that all of project looks more standard and share the same set of configurations like workflow, workflow schemes, issue schemes, issue type, screen schemes.
Pawel: So does that include also the fields?
Irene: Yes, the field configuration, yes. With this new agreement, we started looking for tools to help migration from server project to cloud on a per-project basis. So Pawel, in your story, you use the Atlassian Projects Import Utility right? So we looked at that but again, unfortunately this tool only imports on the server side, from server project to another server project instance. But we want a tool that actually moves from server instance some projects to the cloud instance. So this tool we are not able to use, we are not able to use that tool.
Irene: We searched the marketplace and we found one app called Backbone Sync and it seems like it would do what we want because in this app, it actually can move data from server instance to the cloud instance. It doesn’t matter if it’s server or cloud, it just synchronizes the data. But in order to do that because this app can only move data, it doesn’t migrate project configurations and doodles so that means that we have to prepare the project on the destination cloud instance. So we have to create a generic project before we synchronized that data using Backbone Sync.
Irene: Another advantage to using this second approach that I wanted to mention is that it turns out that we can minimize the downtime. We were talking about downtime, it’s very critical to customers. Using Backbone Sync on the Cloud site, we can minimize that downtime because users can still work on whatever project they are working on and so we create a new project, empty project, and then we do the sync, only loading a new project or Cloud instance.
Pawel: Okay so what were your next steps for the migration?
Irene: Yeah, so just like in your journey, we always need to do some backup, we need to do some mapping. We need to do the field mapping, workflow status, sprints, boards and like I already mentioned, we had to create an empty project in the Cloud instance before we do the data sync. Of course, we also will also test with a trial instance.
Irene: So Pawel, you mentioned in your journey, you said you were able to extend your trial instance. Before the cloud trial earlier, it’s only seven days so it’s nice that Atlassian extended later on to 30 days. I just wanted to let the audience know that being a Solution Partner, DragonAgile is actually able to help customers extend to a few more months so if you need more time to test out your trial instance, we are here and happy to help you out so you can just email us.
Pawel: Now, out of curiosity, how did you deal with the five user limitation?
Irene: That’s a very good question so on the server side, our customer already has a lot of users on the server side and manually adding on the Cloud side. Since those users are not already existing on the cloud, it’s labour intense. It’s a really long process. For that, we have to import. The procedure we used was importing all server users to Cloud first so Backbone Sync can map assignee, reporters, commenters to the right person so the user has to be there first.
Irene: With this limitation that you asked about, we have to ask Atlassian to help us so that we could do bulk user importing with the correct status and Atlassian actually replied very quickly, was very helpful so we got this part done. After that, we started using the app to do the project merging, importing.
Pawel: So your strategy would have generated bunch of notifications to those users. How did you resolve the notification problem?
Irene: That’s a really good question! We thought about that, people don’t want to be bombed by thousands of emails. Just like in your migration procedure, there were a few steps, we have to be thoughtful before we do the migration, so notification – it’s one of those things that we have to think about. We actually created one special notifications scheme called none notification scheme so that we turn off all notifications. So when we create an empty project, we associate first this notification scheme so that nobody, maybe only the admin just to monitor the issue that are created, but everybody else we’ll just turn them off using this none notification scheme.
Irene: One other thing I wanted to mention from my migration journey was after we complete moving limited number of projects from server to Cloud, so we still have a lot of remaining projects on the server side. So we come out with a solution – using a second cloud instance to archive the whole instance from server just so that the customer can refer back to some old projects easily. Remember that ideally, they want to move out totally from server to Cloud because they don’t have resources to maintain and it’s been hacked multiple times. This is a second cloud instance will be used just for archiving purpose.
Pawel: Can you share any additional issues that you’ve encountered?
Irene: We had a couple hiccups. While doing the migration, we realized using Backbone Sync, all the users need to be active in order to map comments, map assignee and the reporters so that means even those users on the server side, it’s inactive users, we had to enable them first and then do the migration. After that, we disabled them again because we don’t want the customer to pay a huge bill just because of this migration. So that’s the extra steps we have to take care of. If we had done that it’s also quite labour intense. Again, we also ask Atlassian when we finished, we provided the Excel sheet with users and their correct status like active, inactive. Atlassian was able to quickly deactivate those users.
Irene: One thing you mentioned was duplicate field names right? That could also cause trouble when you’re not taking care of because we can easily pick the wrong field. Just because they have the same name right and that will cause the migration to fail.
Pawel: Sounds like you wanted to clean up a little bit more thoroughly right?
Irene: You always need to do like more thorough cleaning. There will always be some little things you can miss – that’s why documentation is very important. Again, after migration, you need to optimize the system, like say, we already create a generic project template. We need to deactivate those zombie active users and previously, we use the none notification scheme – now we have to enable the correct notification scheme associated with the project so other projects will work properly.
Irene: Even though we cleaned up in the preparation step, we still need to double check; further clean up unused fields, status, schemes, that’s a good practice. Testing and verification is a very important part. You want to know if all of the migration is actually completely working. So as an admin also being a Solution Partner, helping customers out, we will do the verification after the migration definitely. Pawel mentioned that user knows best that because there are so many users. Everyone uses Jira in their own way so you never know how they use it and know what issues will come out.
Pawel: Verification is very important, especially in a test environment. Sounds like our migration stories have many similarities, although we use different ways of accomplishing pretty much the same goal. We were both migrating projects from somewhere else whether it was Jira Cloud or Jira server into another Jira Cloud instance.
Irene: In your story, you also mentioned that you were thinking about using the Jira Cloud Migration Assistant program right? So why don’t we take a look? You were not able to use it but definitely, this would be a tool that Atlassian is actively working on that and hopefully it’ll help make anyone’s migration story smoother. Let’s take a look and see what the options are.
Irene: So I grabbed this slide from Atlassian. This will be a timeline to see what will be available when this migration assistant will be available. It looks like right now, there are some features in the early access program.
Pawel: This is exactly what I was looking at, the early access program. From this, it looks like there’s a lot of new things already, we might have considered this avenue instead of the one we did!
Irene: It seems like a better version will be available in early 2020 then by mid 2020, hopefully at that time we have something that will help everyone, Atlassian can have something that can help everyone. They can minimize the painpoints of this migration journey.
Irene: Let’s see what’s the current features are, Pawel. So it looks like there’s users and groups that are being covered. Project details, all of that is outlined there, like project roles and basic workflow stuff is being migrated. Screens, status, categories, permissions and issue security. So it looks like that there’s a lot of stuff that’s been migrated already.
Pawel: Let’s look on the issue side. On the issue side, it looks like all the basic fields also be covered. You can see all the assignee, reporters, even some custom fields and the labels, dropdown, checkbox. Looks like also the important agile things are being migrated as well which is quite important for a lot of people including epic links when I was looking at the early access program, that was our showstopper. Looks like they’ve updated which is good news.
Irene: You would have used it if it was available, right?
Pawel: That was one of our showstoppers.
Irene: Let’s look further at some data that isn’t migrated. So for the project setup, it looks some advanced workflow features like conditions, validators, post functions. But the situation can change, Atlassian is gathering feedback and improving that. Hopefully, it will be available sometime later.
Pawel: But this would be perfect for somebody that has a Jira instance without an administrator. Usually they just build workloads without those things so it would be useful and not a big issue. For us, we use a lot of advanced features so it was better to go with Jira Server. Let’s have a look at what else is there.
Pawel: Looks like the boards, filters, names – that’s also something that was not migrated using Jira Server migration method but I recreated all of that stuff through the back. It would be nice if this was migrated, especially filters and boards because users love the filters, they love searching for stuff.
Irene: Take a look at this slide, it looks like Jira global entities may not be covered like global permissions, dashboards, cross-project boards, filters and of course the app data, this will not be covered definitely. It’s up to the app vendors to provide their own tools or methods to migrate app data. That is probably one of the biggest pain points for every migration.
Pawel: These plugins – some of the vendors don’t even have instructions on how to do it so to reach out to them directly. Whether it’s Jira Server or Jira Cloud, it doesn’t matter. Migrating app data would be a great benefit for everybody I think.
Irene: To summarize, it looks like there is no magic tool with one click just to migrate everything. We wish that there is one tool. Life would be so much easier! No matter which approach that we use, it looks like we always need to do lots of manual preparation, clean up and of course, verification after migration. This will always be part of everybody’s migration journey.
Irene: Like I’ve said, DragonAgile is a Solution Partner will be much happy to help if you need a hand with your migration, feel free to contact us!
Pawel: Cloud migration is definitely a journey and it does take a lot of time and preparation and organization and if you don’t have resources to do that, DragonAgile would be a good choice.
Irene: Thank you! That’s pretty much that our presentation and now it’s time for taking some questions. The first one is how do I contact Atlassian for support on cloud migration?
Irene: So you can contact Atlassian for support by opening a migration support ticket at Atlassian support. I believe if you put in the keyword migration, it’ll raise to higher priority for Atlassian. Probably you get the reply pretty quick.
Pawel: And I gotta say, I’ve reached out to Atlassian migration support and it was pretty fast. Compared to everything else, that’s like unbelievably fast – within a couple hours I already had my answer.
Irene: That’s great! It really helps, some people talking about how it takes few months, it’s really a long journey. So another question we have is how long can I keep an unlicensed server with our archived projects for reference?
Pawel: As far as I know, your Jira server licenses is perpetual so you can basically keep it as long as you keep server up and running. They just don’t get any latest updates – you’re just sitting on that version of whatever you have. You can keep it as long as you want to – it’s a Jira server so you’re just paying the bill for keeping it running.
Irene: What was your backup plan in case things were not migrated properly?
Pawel: I’ll take that one. For myself, because it was our target and sources, one of the instances was in Jira Cloud, we basically cut the cloud running until it was ready to be released. I have created a backup of our Jira Cloud instance using the Jira cloud backup, not your server backup for Jira Cloud and basically what happened was I kept a backup separate on a separate secure space so that we can restore back to it and we would have just basically told users (this was happening over the weekend) on Monday morning that, hey everything’s back to the way you are and the way they were. We did perform the migration, we’ll have another test.
Irene: Our last questions is what is the biggest pushback from your customers?
Pawel: Lack of app data was probably the biggest thing, especially for things that users use on daily basis. Biggest thing is JIRA testing so Zephyr data was really important to my customers, by customers meaning my entire organization. App data is a critical thing and it is extremely painful right now to migrate it and like you said, Irene, a lot of the vendors don’t even provide instructions on how to do it. Like you said, it would be nice to have. We don’t so that was the biggest pushback. That seems to be pretty common across all customers.
Irene: That’s all our questions. So we will make this presentation available on our website and we’ll send out the link to the audience. If you want to check back or refer this to somebody else, feel free.
Pawel: If you have any other questions, feel free to email either one of us.
Pawel and Irene: Thank you guys!