Transcript
not the first event on the community, the first event globally. This is the first time that we are organizing an event on this topic. I am Federica Saba, I am the Global Training and Certification Manager. Basically, I am in charge of bringing to the market all the nice news about education. We have been waiting for this event for quite some time. Today I am super happy because we have gathered some great people. We have the new leaders of the VMCE Plus group in the Veeam community, Shane, Yoss, Elo, Elo, and we have a special guest, my friend Jessica Marshall from the certification team, who of course took part in shaping and building the VMCE Plus certification. Jessica, if you want to just introduce yourself. Thank you, Fede. Hello, everybody. It's so exciting to be talking to so many different people from all different parts of the world today. Jessica Marshall, I am in the certification team, like Fede said, and I am responsible for developing Veeam's professional certification exams. A bit later, I'm going to go into what that process looks like, why I have a job. But I've been with Veeam for very nearly eight years now, and our programs have been through a lot of changes. I'm really excited, I'm really proud to see our programs develop into what they have today. It's a great progression, and so I'm looking forward to talking to you all about that today. Fantastic. Thanks so much. I will... Does it work? Yes. Okay, I am sharing, of course, my screen. We will walk through some new certification roadmap. We will see how we are raising the bar and how we can actually find a way how people can get certified. Of course, we will give the floor to Jessica and to Jos and to Shane on how the certification has been shaped until right now. Let me just give me just a second. I have a technical little issue here. The nice thing of the direct. Okay, so sorry for that. And then I was saying that we will listen from Jos, from Jessica and from Shane how the certification was shaped. Hopefully, we will listen to some new secrets behind the process. And finally, we will share some useful resources for you. So certification VMC plus is live. This is the situation that I can believe it was one year ago. We used to have... It is actually quite impressive what we have done in the last year. We used to have the Veeam Backup and Replication training. Then that gave... I am hearing some sound. If you can just mute yourself. We used to have the Veeam Backup and Replication training. And then with the training, we used to go to the VMC certification. And then we had the design architect training that used to brought you to the VMC certification. So what happens there? We removed the VMC certification. We removed the VMC training. We removed the VMC certification. And from the old program, we just keep the Veeam Backup and Replication instructor. What happens after that is we launched the Veeam Backup and Replication training and we launched the old stack of training that now you see on screen. Of course, now, if you want to get the VMC+, you have to take the Veeam Backup and Replication first. And then you have to take the Veeam One and the Veeam Orchestrator training. After you earn the VMC certification, the VMC certification, the next step is taking the enterprise data security training. And once you have both the VMC certification and the enterprise data security training, you will become eligible to take the Veeam Certified Security Expert that we will launch very soon. We will also offer a new certification badge for architects that's in the roadmap. That will be invite only. We do not have a launch date yet. Just stay tuned. I'm sure that we will inform you very soon. What's important for this call is this. Again, to access to the VMC certification, you will have to take the three trainings, the Veeam Backup and Replication, the Monitor, Manage, Analyze, and the Sky Automate and Secure. Those are the three requirements to be eligible to take VMC plus certification today. But what happens if you are already certified or what happens if you are already trained? So if you are already VMC certified or if you are even trained in Veeam Backup and Replication in the version 12 or 13, you can skip the first of the three steps. So you skip the Veeam Backup and Replication and you start from the Monitor, Manage, and Analyze training. If you have never been trained or certified in Veeam or if you are VMC trained or certified in a version older than 12, so 11 or older, then you have to start from scratch with the Veeam Backup and Replication training as your first step. But how you want to access to this new stack of training? We have recently launched the e-learning subscription as an SKU. It's an SKU you can purchase like any other product. It's available through the Veeam channel. So if you are a customer, you go to your partner. If you are a partner, you go to your distributor. Once you purchase the e-learning subscription, your people get access to the full stack of the Veeam trainings. It's available, of course, 24 for 7, for 12 months, and you always have access to fresh content. So if a new version of the training is released during your contract, you get the new content, too. So the question would be why we are doing this. I think this was the missing piece for our attendees to be fully prepared on the old stack of Veeam Enterprise products. With the older program, with the Veeam Backup and Replication, we used to cover the foundation, and we used to cover it very, very well. But you know very well that the platform has evolved, has grown up, and Backup and Replication only was covering the foundation of our platform. With this new training, we cover the old stack, including the advanced and the premiums. We now cover, I don't know, cybersecurity, monitoring, analytics, orchestration, the compliance. So this is a real game changer. We have built a clear path so you can grow step by step at your own pace and always with the latest content in your hands. But what do you actually gain by taking this certification? And I have asked, we have surveyed our students. We have asked them what the training meant for them. First of all, I didn't even need to ask. By taking this new training, you can be among the first to earn the Veeam CE Plus certification. We change our programs after 10 years. So this will be a real differentiator for you. Actually, I am quite impressed of the people that are publishing on LinkedIn and on social media about their badges. There is a lot of people already publishing about having taken the Veeam CE Plus. But students also told us they gain new skills. They improve their process. They have shown a real impact in actual customer environments. Almost 80% of the cases, they say they gain confidence. They gain credibility. Certification validates expertise, reinforce the credibility with peers, with customers, with possible employers, of course. And for organization, the perception customer has of them with better modern services. And this certification also helps to stay, of course, current with not just Veeam technology, but also with the technology of the market that surrounded us. And of course, today, we wanted to give a little different and hopefully a little interesting approach and discover how actually the certification is built. I was provoking a few days ago on the LinkedIn asking, do you think the certification questions are just randomly selected questions? And to answer that, of course, I brought the expert, which is Jessica Marshall, one of the leaders of the certification team, who really crafted DMC from scratch. And hopefully, she will tell us some secret behind the scenes so that everyone also understands how much work. Believe me, there is much work. There is a lot of people behind, people like yours and like Shane and a lot of more others that contribute to this certification. And let me tell you something. The Veeam community always has a big, big role in this process. We have beta tester, beta certification taker. People have been crafted the new certification, like in this case. The Veeam community is always the place to be. And yeah, nothing else. I think, Jessica, do you want to take from that and explain a little bit more about this process? Sure, thank you, Fede. OK, so like Fede said, the process for putting an exam together is not as simple as somebody sitting in a room and writing a bunch of questions and putting them out there and seeing how it goes. As I'm sure everybody on this call is keenly aware, the stakes can be quite high for a professional certification. So we have a whole process that we go through that makes sure that the exam that we turn out at the end of it really measures what it's supposed to measure and is a valid exam that we can stand behind. So what does that process look like? Obviously, it's a professional qualification. So the obvious first step is to define that profession, what that job role looks like, what job role are we trying to credential here? And then what does competency look like in that role? So what can people do? Who is competent before we give the seal of approval from Veeam that these people know what they're doing and we're happy to stand behind them? Because that's what this credential represents. So the first step is to conduct a job task analysis. Now, this is a fairly gruelling, in-depth four-day... For us, it's about four days. It usually lasts about four days process where we gather a group of subject matter experts. So these are people that have already proven themselves competent in these roles of experts in the field. We gather them together and we grill them for four days about what the job role looks like and what being an expert, what being competent in it looks like. And then by the end of those four days, we have mapped the job role, we've mapped the levels of competence that we expect, and we've turned that into an exam blueprint that has all of the topics, how many questions need to be asked on each topic, and even goes into the details of how cognitively complex those questions should be. So how much they should make your brain hurt while you're in the exam. So that's all decided upfront by these experts that we put a lot of trust in. And that's, as Fedi said, people from the Veeam community, people who are proven to be experts in this field. That's the exam is all decided upfront and mapped out in great detail at the start during the job task analysis. So then the next phase is actually to write the questions because obviously that does have to happen at some point in the process. It's just not all there is to it. And that is actually a surprisingly longer and more rigorous process than a lot of people expect even in and of itself. So again, we gather a panel of subject matter experts. We quite often use the same experts as we do for the job task analysis because they've already gone into huge amounts of detail about what this exam should look like. So they're usually set up well to then actually write the exam. Although we do sometimes recruit more people. Again, this is experts that have proven themselves within the Veeam community. So we recruit people to write the exam based on their technical expertise. We don't recruit people based on their experience in exam writing or their knowledge of how to write questions. And because of this, the item writing, the exam content writing always starts with a briefing. And guidance, fairly detailed guidance on what makes good questions. How do you write good exam content? And once all these needs have been briefed on that, they will be guided throughout the rest of the process by people who are experts in the field of creating exam content on how to write good exam content that works, that people can understand and that deliver the results that we want it to deliver. So we have the expertise from the SMEs, the technical expertise. We have expertise on exam writing and that comes together in a very collaborative process. So the SMEs are given assignments to fulfill. Once these assignments have been fulfilled of exam content questions they have to write, SMEs and the test writing experts meet together in collaborative workshops daily. Again, this is involved and quite grueling and they review the questions and it's quite an in-depth review process. Does it test what it's supposed to test? Is it technically accurate? Does it adhere to the item writing best practices? Is it at the right level of cognitive complexity? Is it testing to the depth of that it's supposed to be testing? And every question is reviewed by the group and modified as needed. And then finally, if it's a good enough question it makes it through the process. After that, it goes through an editorial review and then the exam content has been created and it's ready for the next few phases because there are a few more phases after this that I'm gonna touch on in a minute but we have exam content ready to go at this point. So as I said, we have trusted experts working on these phases of exam development. Two of those experts were Shane and Joss and they did a fantastic job defining and writing the exam that's out there today. So guys, I'm gonna hand it over to you for a bit if you'd like to talk maybe a bit about your experiences. Yeah, sure. Shall I start? Yeah, go ahead. So first of all, it was a big honor to get involved in the whole process because we all took other exams and we all know that there is a different experience about the questions and the results of all that. So it was really an honor to be part of the process to be able to create an exam with the proper questions or at least questions where we have some influence in. So it was, yeah, glad to be the part of this group and the whole group together was a nice group of people. So different, so many backgrounds, system engineers, pre-sales, architects from Veeam but also a lot of people from the community that works with vendors and all had their different experience with the products. And then we started with the whole process that Jessica described and I wouldn't have imagined so much theory behind the whole process and how much effort it took to create an exam like this. It started with the process of the job task analysis, right? So where should someone who take the exam, what's the level when he should pass the exam? What's not to need to have, how much experience? The whole process was so interesting. I know Shane, have you anything to tell about the job task analysis part? Yeah, thanks Jas. So as Jessica described, we initially, we met for about four days and you may be asking like, what in the world can you do for four days just to come up with exam content? Well, that's not exactly the gist of what goes on. First of all, we have to define the candidate, the minimal qualified candidate, the MQC as we call him or her. And what do we have to talk about for that candidate first? Well, first we have to talk about what foreknowledge, in other words, what knowledge does that candidate have to have to begin with just going into training or does anybody have their sound? I think it's yours, go ahead. Okay, okay. And so the foreknowledge that a minimally qualified candidate has just before they start to tackle their learning journey. And then we talk about the qualifications and the knowledge that the person has to have to begin their journey or before they take the exam. And then we need to decide how much experience with Beam Data Platform products does that person have? So like three main areas, and that's just the beginning. Then we actually, in the job task analysis, then we actually move forward and carve out the competencies and skills that needed to go into the task and knowledge required to take the exam. And that's done not just with one product as was in the past, as Federica mentioned, but with now three with VBR, of course, V1 and VRO. And then after we mapped out those skills, then we kind of bring them all together and what you all know, if you've taken certification exams before, what is culminated into an exam blueprint. And then after that, as Jessica mentioned, we meet to discuss questions. We actually meet one session to go over all the questions and we have to divide all the tasks and skills and knowledge that we came up in the exam blueprint. We have to divide that out into what we call competency levels, which are different levels of cognitive understanding of the product based on, how do we call it, metrics and tiers. We actually had to rate the, oh, what was it, Jessica? Was it how often or how critical certain tasks are? And based off the criticality level or how often the MQC performs certain tasks or functions or features, then we came up with cognitive levels. And there's three of those, what we call remember, understand and apply and analyze and evaluate. And yeah, analyze and evaluate is the hardest level. So when you go into the exam and you see some questions that may be just easily recollection type questions, that's what we call the easiest, not necessarily easy. If you don't know static, what I call static information about being data platform, it's still gonna be hard. And then it kind of goes up from where they're from understand and apply and analyze and evaluate. And then once we assign cognitive levels to all the tasks and knowledge level required for the exam, as Jessica mentioned, then we come up with questions and we spend the bulk of our time. And that's actually the first meeting of the question development process. And we take a break and write the questions. And then we actually came back and discussed the questions and really, really put them through rigorous criteria. They had to meet obviously knowledge criteria as far as does it meet or does it align with BEAM Help Center user guide information, best practices, KBs and just general use information. Are the questions concise and to the point? Do they ask what they're supposed to ask? Are they neutral, regionally, cultural, et cetera? And do the questions actually ask what they're supposed to represent based on the task or skill set within the blueprints? And then a whole bunch of other criteria that we had to make sure the questions evolve around. We were given guidelines and documents on how best to write those. But anyway, all the SMEs, about 15 of us wrote about 15 to 20 questions. And then we had to go through each and every one of those questions and made sure all the questions meet all those criterion. And then as Jessica mentioned, when we were finally done with the IDW phase, then it had to go back to the certification team and they had to go through them again. And then it was piloted. And then, well, what we've come to finally realize and why we're having this session, is the release of the certification exam. So I hope the work Yas and I, and I know a handful of others are on this webinar with us today, all the work that we put in, hopefully you guys find it useful, beneficial, and rewarding to take and hopefully pass the exam. Jessica, Federica? Yeah, I mean, you guys were absolutely brilliant to work with and it was a real, Yas, you said it was an honor to work and it was an honor to have you guys working on it for us. And you did such a great job. It was huge amounts of work and yeah, you showed up and you did that work and you did it so well. So thank you so much for all of the hard work that you put in. As you touched on there, Shane, there are a few more phases before we can release that exam. So we, in the case of VMCE+, we did pilot it. Pilots are contingent on us being able to find a large enough candidate sample, but we did pilot the VMCE plus exam. And the whole point of doing a pilot is to test whether, so we collect statistics on the questions from administering the test in a real live test environment. So we can test out the questions. We can see how well they do, things like how many people got it right? How many people got it wrong? Were the people who were scoring highest also getting that question right? Or is there a discrepancy there? Were any of the distractors never chosen? Were some people choosing a distractor? Were the majority of people choosing one particular distractor because it was actually maybe correct and it had slipped through? So it's all about verifying that the questions that were written do actually function the way that we expect them to function in a test environment. And we also gather statistics that then contribute towards the setting the pass mark because the pass mark is not set as an arbitrary 70%. It goes through the pilot. We use data from that. And we also use data from another process that happens after or in tandem with the pilot called standard setting. We get a new group of SMEs who have not seen the questions before to review them, evaluate them on their difficulty levels for the minimally qualified candidate, that person that Shane mentioned they have identified and described in great detail from the job task analysis. That is the person that is, it represents the person who is competent to the level that we want certified people to be competent and no more and no less. So evaluating all of the questions in light of that minimally qualified candidates, that data on how difficult those questions are then gets, that's reviewed and analysed and it's used to set the cut score in conjunction with the pilot data at a place that is meaningful and strategic to make sure that people who pass are genuinely competent and people who can't pass are genuinely not at that level yet. So that's the final process and verification, pilot, then standard setting. And then it goes on to publication. Obviously we publish through Pearson VUE. There's a lot of checks and quality assurances to go through with them to make sure that when we actually deliver it all of the questions are scored correctly and that it functions as it should do. And then, ta-da, the exam is live and you can all go and take it. Hooray. And I have a holiday. Finally, right? Ish. A well-deserved holiday. Very well, very well-deserved holiday. So Jessica, the exam is now live. Someone actually take the training. They will go, I will share another slide at the end, but they will go to the certification page on vim.com. And if we recognize them as eligible, what happens there? They will be breached to the Pearson VUE. Can they take the exam in a physical or in online? Can you let us know a little bit more about the certification process? Sorry, about which part? Well, what happens after you are in the vim.com page, you are recognized as a person that can take the certification. So your profile will change. You can click on a button on the vim.com page and you will be breached to the Pearson VUE page, actually, where you can take the exam. So the exam is available online. It's available, can you take it online? Can you take it in person? Do you have to go to a person of your center? How things work? Well, yes, as you described, you get redirected. Once you go to vim.com and you click on the button to manage your exam, then you'll be redirected to Pearson and you can book your exam. Definitely, you can take it either online using Pearson VUE's, sorry, Pearson Professional Assessments. They've just changed their name. Yeah. We'll carry on. It's like when they changed Twitter to X, it took me like two years. It'll take a while, yeah. Personal access, yeah. Sorry about that. Go ahead. You can take it online with their proctored services or you can go to, you can find a local test center and book yourself in for an exam there. Just a note, when you do take the exam, scores are not released at the end of the exam. You will receive that in a separate communication after the exam. This is an industry standard. Not everybody does it, but it is industry advice not to release scores directly after the exam while people are still in the testing room because it can be very disruptive. So you receive them separately. We also don't do, I'm just anticipating questions that have come up or that may come up now, but we don't do section level scoring. We don't report on that. That is again, industry advice because of the way the exam is configured and the way that it's set up. And because we use scaled scoring, the scores can be very confusing to the candidate when they receive them. They can give what seems like very contradictory advice. So they're not very useful for studying, but also for exam security purposes, the general advice is not to release section level scoring in case people have been harvesting our questions because we don't want that and it could give away the answers. I'm glad you had mentioned the change on when the timing of the score, because that changed, I wanna say version 11 probably. Before that it was instantaneous. And maybe it wasn't version 11, maybe it just started with 12, but I was like, where's my score? Because previously it was instantaneous, but yes, I can understand that when you're, if you're doing it in person and before the, a couple of times ago, I did and I liked it that way. I actually went to a center and took it, but recently I like it better online now, but I can understand you going, you know, I passed or, oh my gosh, I can't believe I failed and causing disruption within the center of others taking other tests. So interesting to hear that. Thank you for sharing that information. Yeah, fantastic. And Jessica, I know it's not the moment maybe to ask, but can you tell a little bit more about VMCSE that is coming? Anything that you want to share to this group in advance? Anything, you know, something about when the next certification, how it would look like, anything that you want to share with us? Okay, yeah. So the exam is actually, it's been through the same process minus the piloting. There is no pilot scheduled for the VMCSE exam. Just gonna mention that right now because we, like I said, it's really contingent on the available candidate sample that we can get hold of. So we have taken a different approach with VMCSE. We are relying a lot more on the standard setting process, the ANGOF that I described earlier about evaluating the difficulty of the questions and question review. So we're relying more heavily on those, but it's been through a very similar process just minus the pilot. It is published with Pearson VUE. It's ready to go. It's finalized, but we have a few things to sort out on our systems and we will be releasing it just as soon as we can, but I don't have a definite release date just yet. So we're working on. Sorry, Federica, Jessica, I'm not sure if you saw the question from Yasin. Sorry if I didn't pronounce your name right. They were asking about in-person learning or trainings for VeeamONE and VRO, if and or when you are aware when those will be available for version 13? Yes, but yeah, soon-ish. Yeah, coming. Veeam Backup Replication, as you know, is still available as always in our network of training center. All the information is on veeam.com on the certification page. So you can still access that content with the official Veeam training centers. Regarding the VeeamONE and the Veeam Orchestrator, it is in the roadmap, but we do not know yet when these will be available instructor-led. There is not a date. We have to prioritize other topics that you will hear about these new topics very soon, hopefully, so there is something new coming. But for the moment, we do not have a go-to date for the moment. VeeamONE and Veeam Orchestrator, as well as Enterprise Data Securities is only available with the e-learning subscription, which is a great tool, by the way. So I would just, I would really investigate and know more about the e-learning subscription. We took a lot of effort, as you see, for the certification, but a lot of effort also was done by the team that developed the trainings, so. Great, thank you, Federica, appreciate it. Yeah, there is something else that I wanted to share for those of you who are partners. The VMCE plus certification is part of the compliance. If you go to your partner portal, that's updated there. So you will see that certification is required to maintain or for grow your level with Veeam, so. Nice information for those of you who are partners. So I do have a last, a very last slide that I have prepared with the, I don't know, let me know when you see my screen. Is that visible? Yep, visible, thank you. Yeah, so if you are decided to take the certification, these are some of the resources you would like to use. I know there are a lot of QR codes, but I thought that the deck, the PowerPoint that contains the links was not available to the attendees. So you will see on this screen that you have the Veeam.com page. Here you can see all the information about training, certifications, how to access, et cetera. Those of you who are partners, you can also have access to these same content, but also more trainings that we develop to give a better sales and resale service. Of course, the place to be is the Veeam community and inside the Veeam community is the VMCE plus study hall. Thanks again, Joss and Shane for leading this group. You have here a lot more of information if you want to book your exam. It is actually the same page, but of the Veeam.com with another, is another profile. And yeah, and that's, I guess, everything I wanted to share after this call. I don't know if you guys want to add something more. I just, I'm going to put something in the chat for all the attendees. For those not aware, I posted or created a post about talking about the JTA. Well, that was kind of tongue twister. Job task analysis and the workshop phases and a little bit more, kind of what was discussed today about the certification development process. And I will put that in the chat, the link of that in the chat. And then if you want to keep up to speed and up to date on pretty much anything VMCE plus, Joss and I moderate the VMCE plus study hall page, I'll put the link to that page on the community hub, also in the chat. And you can, if you're not signed up on the community hub, you can register there and then choose to subscribe and follow that page. Joss, do you have anything? They, yeah, yes. Again, it was a great experience and thank you Federica and Jessica for the session to explain the story behind the exam. And maybe it's a good idea to do after the summer period and another webinar with some test takers and their experience of the exam. It would be a nice next topic, I guess. Yeah, that's all. Yeah, I would love it. Yeah, nice stuff. So thanks so much, Jessica, again, for participating in this meeting. Thanks again, Joss and Shane for leading the VMCE plus group on the Vim community. Thanks a million to Safiya and Madalina for all the support of this event. Don't leave the, don't stop here, leave your comments on the VMCE plus study all group. We all will be around to respond to your question and anything that you need to know. And we all see you soon in the community, especially in the VMCE plus study hall. Thanks everybody, appreciate you joining. Bye-bye. Ciao, ciao, ciao, ciao.
