Note This document is based on the original spoken Support WebCast transcript. It has been edited for clarity.

Wally Mead: This session is on "Microsoft^® Systems Management Server Operational Procedures." What we're going to do in this session is discuss and introduce Operational Procedures for you. So our session agenda for today is an introduction to the SMS Operational Procedures. There is actually too much information to cover in a single slide, so I'm just going to give you an introduction and point to where you can follow to get additional information on the topic.

So what we're going to do is introduce something called the Microsoft SMS 2.0 Operational Template (slide 2). We're going to talk about how you acquire the SMS 2.0 Operational Template. We're going to find out why do you need it. Where in a deployment would you use the Operational Template and the information provided in it, and how can it help, regarding your return on investment. That's basically going to be the agenda, the SMS 2.0 Operational Template and the operational procedures that are contained in that template.

Just as a quick overview, the Operational Template includes a set of best practices for SMS operations. I'll talk about how this was developed a little bit later on in the presentation, but it is a set of best practices for day-to-day, weekly, and monthly operations of SMS environments. So it's going to introduce for you the tasks to be performed, what tasks need to be performed to maintain your system and ensure that it's operating optimally. And we'll also talk about when those tasks should be performed. Should they be performed on a daily basis, a weekly basis, et cetera. We'll talk about who should perform each of those tasks. We'll talk about what tools will be used to perform these tasks. At the end of the presentation, we'll certainly have time for Q&A.

Some of this information is going to be documented in the Systems Management Server 2.0 Administrator's Guide. For example, in Chapter 18, there's a good section on maintaining SMS, and there's a section on performing maintenance checkups. That talks about some of the daily, weekly, and monthly tasks that you might want to go through and employ in your environment. Also in that chapter is maintaining SMS databases, where it talks about backing up your site, some of the autodeletion functions that we'll reference throughout the presentation, and performing some SQL Server™ maintenance tasks. However, not all the information that's in the SMS 2.0 Operational Template is documented elsewhere and widely available to you. So that's why we're going to introduce the Operational Procedures Template for you.

So, an overview of the SMS 2.0 Operational Procedures (slide 3). What is the Operational Procedures Template? Why do you need it? This is a document that describes methods to deploy and use SMS in an effort to minimize your downtime and maximize your effectiveness, in other words, your return on investment. These are two very large goals for an SMS administrator and his or her management, is to maintain the SMS environment to ensure that if at any time it goes down, it's down for as short a period of time as possible, with the highest amount of uptime as possible, and to make sure that it is running as effectively as it can so that you can require or achieve a higher return on investment. So the Operational Procedures Template will give you the information, the procedures, the staffing models, et cetera, that you need to ensure that.

How do you get the SMS 2.0 Operational Procedures Template? It's available as part of an SMS service offering from Microsoft Consulting Services. I will introduce a lot of the information to you, but again, there's way more information than what can be covered in a one-hour presentation. So I won't be able to cover it in full detail for you. You're really going to have to engage MCS if you want to get the full benefit out of this.

It was our MCS practice in the United Kingdom that developed this service offering for their customers. So working with their SMS 2.0 customers, they developed this set of best practices that have been employed in the United States and throughout the world, and have proven to be very, very effective.

The Operational Procedures Template is just one component in a set of components in the SMS service offering from MCS. I'll just give you a quick rundown of a couple of their offerings in the procedure itself. They also include an SMS security document. This is a document that describes the different security considerations for your site — somewhat similar to the "SMS Security Essentials" white paper (http://www.microsoft.com/smsmgmt/techdetails/secessentials.asp) that's up on our public Web site. It just happens to be a smaller version of that document. It was out prior to the SMS Security Essentials document.

It has a design template that helps you go through and design your site, it helps you see all the different design considerations that you might want to go through. It has a design checklist, things that you should be validating through your design to verify. "Hey, in the best practices, we've seen these things being very important to consider during the design phase." It's just a checklist to make sure you've looked at all those features.

It has a project plan, so you can lay out your timelines on each of the different phases — the planning, the testing, and the deployment, to help you with that. It has risk review, areas to look at for potential failures in your environment, including training of your new employees, or employees on the new SMS environment. It has a test plan, so if you want to go through and perform a test pass on SMS 2.0, or in the future, Topaz, prior to implementation in your environment, what are the different areas to test? How do you go about testing those different areas, and what are the features and the results that you should be looking for? All that information is documented in the SMS service offering, as well as the Operational Procedures Template.

Again, the Operational Procedures Template is a set of best practices for SMS operational procedures. It covers two primary areas of SMS: change and configuration management, which is primarily what SMS is there for — maintaining, upgrading, controlling what's present on server and client desktops — as well as problem management, and the actual problem resolution. So when you do encounter a problem in your environment, how do you go about solving that problem?

Through numerous different customer engagements, the SMS product group, Microsoft Consulting Services, and third-party consultants talking to us, we found something that's highly advantageous for you, and I want to make sure everybody understands that. I think I've mentioned before in other presentations I've done. But what we've found out is the more time that customers spend planning their implementation prior to actually performing their implementation process, the more successful their deployment is.

So at numerous different times, when talking to customers, we found out that the more successful SMS deployments are the ones where customers have spent more time in the planning environment. So they've spent more time in the planning process, planning their environment, testing out the processes, testing out the features, figuring out what the return is going to be on those features, and how to implement them properly.

The sites that are less successful with SMS are the ones that just haphazardly run Setup off the CD and install their site. They just let it run the way it runs, install the way it installs, and don't have any goals in mind or haven't properly planned out where they want to have their client access points, where they want their SQL Server to be. Do they want to use their accounts domain for logon points, or some other resource domain? They haven't gone through a lot of the planning scenarios to really identify what things they want to get out of SMS. So this has been determined to be a large factor in presenting a successful deployment.

The next thing we'll talk about is "SMS Operational Procedures" (slide 4). When do you want to use it? When do you want to get this operational procedure information and deploy it or utilize it at your sites? The slide is a chart that shows you your environment, as far your timeline for deploying your SMS environment. First, you have your planning process. Then you have your piloting phase. Then you have your deployment phase. You can see, between the piloting and deployment phases, the Operational Template.

That doesn't mean that's the only place you could use this information. You may be able to or may want to use it prior to your lab design and test plan. You may want to go out there, and while you're doing your lab testing, find out what you need to be looking at as far as operational procedures, so you can start getting people trained at that point in time.

However, you may need to wait until after you've done your lab design and your test plan, because you may not know what you need to be performing operational procedures on until you've actually gone out and tested SMS fairly well. So it may not be that way in the planning phase, but you want to deploy your operational procedures, or at least have a good grasp of them, and understand what procedures you need to employ and how you want to employ them, prior to the actual deployment of the product.

That doesn't mean you can't derive benefit from looking at this after the fact. I don't want all of you tuning out now because you're already SMS 2.0 customers, saying "Well, this is prior to deployment, and we're already deployed." You may very well be able to derive some better operational procedures than what you're doing now by going through this process — by looking at the Operational Procedures Template, engaging MCS, or looking at the administrator's guide or other available information to determine what are some of the common day-to-day tasks that you need to go through in your environment to ensure that your SMS environment is running optimally.

So that's what we'll look at in this process. You probably want to look at it between the pilot and the deployment phase, but it certainly can be run after the fact — analyze your environment and make any adjustments necessary to improve your return on investment from where you are now.

The Operational Procedures (slide 5). The primary goal of the operational procedures document is to provide SMS administrators with procedures designed to minimize downtime and maximize your return on investment. As I stated before, those are the two primary goals for SMS administrators, and primarily the management above them. It's to make sure that the investment that you've laid out for SMS is working effectively, that you're accomplishing the tasks that you wanted it to accomplish, and that it's not consuming too much of your time as far as failures and having to recover and troubleshoot problems. Giving you good return on investment by the fact that it can deploy software effectively for you. It can generate that asset management information for you. It can help you support your end users. But to perform all those tasks, to improve your return on investment, you need to make sure that you have proper procedures.

What the document is going to do for you is list a series of tasks that should be completed on a daily basis, a weekly basis, a monthly basis, or on an ad hoc basis. In other words, just whenever they need to be done. Some of these aren't going to be scheduled tasks. Some things you're just going to do when the need arises, and that's fine. But there are certain things that you do want to schedule as a daily, weekly, or monthly tasks to ensure that you are following some of the best practices.

Some of this information, again, is documented in Chapter 18 of the SMS Administrator's Guide. However, there are additional things in the Operational Procedures Template, as part of the MCS SMS service offering, that aren't in the SMS Administrator's Guide, and that we'll talk about in this presentation, that may necessitate you going down that avenue.

What you really want to do is tie this presentation together with Bruce Jones' presentation from a couple of months ago (http://support.microsoft.com/servicedesks/webcasts/wc082801/wcblurb082801.asp). Back in late August, a couple of months ago, Bruce did a presentation, "Troubleshooting SMS 2.0 Site Server Performance Problems." That gave some very good details on some of the procedures documented in the Operational Procedures Template. So in the Operational Procedures document there is a series of things that are listed for monthly, daily, or weekly tasks, and Bruce hit some of those in his troubleshooting presentation. What he gave you there were some actual step-by-step procedures, things you need to accomplish.

In my presentation today, I'm just going to give you more of a higher-level overview of how you need to make sure that you're checking your server performance, and how you do that, but not the details that Bruce gave you in his presentation. So you definitely want to take Bruce's presentation and tie it together with some of the information in this presentation, as well as the Administrator's Guide or the MCS service offering, to get you the details on how you perform some of these tasks. So it's going to be a combination effort to get you where you need to be and provide great return on investment for your SMS environment.

The next slide (slide 6) is a representation of how to tie together people, technology, and processes to derive this improvement on your return on investment. In the management universe, there are various different management solutions to help you improve your processes for providing better procedural operations for whatever product you are deploying, whatever that technology is. In this case, we're going to talk about SMS.

If you look at the chart, the little diagram in the middle of your slide, it represents people, technology, and processes. I know it may be hard to read, but if you look at the chart in the middle, it lists User, Desktop, Application, Systems, and Network — different processes and different people for whom you can try to use technology to make improvements. It also shows you different technologies — Helpdesk tools, Asset Management, Software Distribution, and Software Metering. Then some additional items are Performance monitoring, Security management, Availability Management, Event monitoring, and Administration.

By properly matching the correct people with the correct technologies, you can significantly reduce your costs, thus improving your return on investment. Look at the lower left corner of your slide. When the United Kingdom MCS practice went through, they found a Gartner study that said you can reduce your costs by at least 11 percent by following best practices and processes. That's what this service offering is designed to give you: best practices for operating your SMS environment in an efficient manner.

What the MCS offering is based on is something called the Microsoft Operations Framework, or MOF. This isn't a MOF that you're familiar with from SMS 2.0. It's a Microsoft Operations Framework, which is more of a process to successfully tie technology and people together to improve your return on investment.

At the end of the presentation, I'll to point you to some of the additional information that's available to you for further study. I'll point you to the Microsoft Web site so you can gain additional information on the Microsoft Operations Framework, and how it can be used to improve your return on investment. But this slide is just to show you that you can improve your ROI, or reduce your costs, by properly associating or integrating your technology products with your people skills, through some sort of a process. This presentation is on the process of the Microsoft Operations Framework, or the MCS service offering.

Now what we'll do is jump into the Operational Template (slide 7), now that we have that little bit of background. Let's talk about what specifically is going to be covered in this Operational Template, and some of what you can find in the SMS Administrator's Guide and other avenues. So an overview, and I'll get deeper into each of these different areas throughout the rest of the presentation.

The Operational Template is going to cover the following areas: first off, it's going to talk about essential maintenance tasks. These are tasks that need to be performed on a scheduled basis or in some cases on an ad hoc or on a nonscheduled basis, just whenever they're required, to improve your SMS operations. There are over 40 tasks that are documented in the Operational Procedures Template. I looked at Chapter 18 of the SMS Administrator's Guide, and it does have some of the same tasks, but it does not have as many of the same tasks in there. So you have some of the information in the Administrator's Guide.

Next is operational roles, which is the staffing required to maintain SMS and to perform all these essential maintenance tasks. The document goes through and talks about one dozen different roles of staffing members that you want to make sure you have adequate skill sets for.

Last, it covers operational responsibilities. In other words, who is to complete each task, and when is that task to be performed.

So it takes essential maintenance tasks, which are on a weekly, daily, monthly, or ad-hoc basis, ties them together with the people that you have in about a dozen different roles for SMS administration or management, and then talks about the specific task to be performed, who performs that task, the procedures to be performed for that task, and when that task is to be performed. So it ties the 11 or 12 members of your operational team, your operational roles, together with those 40-plus essential maintenance tasks, into the responsibilities of who is to perform each of those different tasks.

Now what we'll do is we'll jump into the first of those three different areas, and we'll talk about the different maintenance tasks that you want to go through and look at employing into your environments (slide 8). First is a quick overview of the maintenance tasks. The maintenance tasks are the essential tasks that must be performed to ensure that SMS is running optimally on an ongoing basis. You want to make sure that SMS is running properly, and that it continues to run properly and as efficiently as possible. These tasks are broken out according to when they need to be performed. There's a set of tasks that you want to perform on a daily basis, a set of tasks to perform on a weekly basis, a monthly basis, and then the as-needed or the ad-hoc tasks.

Some of these tasks, again, Bruce mentioned in his WebCast back in August, and he gave you some of the procedures for those tasks. Some I mentioned in a couple of different WebCasts way back in the very beginning of the year 2000. I think there were a couple of WebCasts I did on SMS site design issues and hierarchy issues, on common problems and resolutions, (http://support.microsoft.com/servicedesks/webcasts/wc031400/wcblurb031400.asp). So I introduced some of those back then. Bruce did some of his own back in August, and we'll look at some of those through this presentation as well.

Now what we'll do is look at some of these daily maintenance tasks that you may need to employ (slide 9). The Operational Procedures Template breaks out the different tasks in the scheduled timeframes.

For the daily maintenance tasks, these are some of the tasks it has listed for you, and this is just a sample listing. Monitor your site status — so monitor the site status of your site server and all other site systems, whether it's your distribution points, your client access points, your SQL Servers, your logon points, whatever. So you're monitoring the site status of your site systems.

Monitor the status of your components to verify that a single component isn't experiencing any kind of an error condition right now, such as the Collection Evaluator or Distribution Manager. You need to monitor your advertisements and your packages to make sure that your packages are deployed effectively to each of your child sites, and at an appropriate number of distribution points at each site.

You want to monitor your advertisements to find out how successful your advertisements have been. How many of your users or computers have received the advertisement? How many have successfully completed the advertisement? How many have failed, so that you need to rerun that advertised program on that computer?

You want to monitor your clients to ensure that your clients aren't experiencing any errors, such as, "I can't find my client access point." So maybe there's a link down somewhere, or one of your client access points has died, or a set of the clients has gone offline because a router is down, or whatever the case may be.

So you need to monitor your site systems, your clients, the components of your site systems, and the advertisements and packages for your sites. Then when you're monitoring, if you do find any warnings, error conditions, or anomalies, things that don't look correct, you want to make sure you fix those procedures or correct whatever the problem is as soon as possible, so that you can return your site to efficient operations.

Another one of the sample daily tasks would be to execute DBCC commands, those database consistency checkers on your SMS site database.

Another one would be to monitor your network performance and network error conditions. So you're monitoring your network through Network Monitor, or some other utility, to determine whether or not you're having any network conditions that might be causing the clients to not be able to find their CAPs; or that cause clients or users to not be able to find an advertised program on a distribution point; or one of your site systems has gone offline, and why is that the case. So what tools are you going to use to monitor your network? And then correct any error condition that you might find.

Another task is back up your SMS primary site. It's an extremely important task, and you need to make sure you're doing that on a daily basis. So that's one of the tasks that's documented here.

There are also other tasks available, and there are many other tasks to take into consideration. The document lists 14 different tasks, as far as daily tasks that you might want to employ. Some of those are combined in my first bullet, "Monitor site status." There are four or five different things there for monitoring. What is important for you to note, as we go through the daily, weekly, monthly, and ad-hoc tasks, is that what we're listing for you is a set of best practices that have been developed through SMS engagements with Microsoft Consulting Services.

That does not mean that your specific environment needs to deploy or employ each of these different tasks on the scheduled basis outlined in the document, whether it's the SMS Administrator's Guide or the MCS Operational Procedures. Each environment will dictate whether or not that task should be performed on a daily, weekly, monthly, or ad-hoc basis, depending on the size of your site, how many sites are in your hierarchy, the number of clients you have in your environment, the number of staff members you have to accomplish or perform these essential maintenance tasks, and what type of activity you have at your site. If your site is only performing inventory on a monthly basis, then you may not have as many tasks to perform as a site that's performing inventory on a daily basis; or if you're only deploying an average of two or three packages a month, you may not have as many weekly tasks to perform as another site that may be performing software distribution on a daily basis or a weekly basis.

So again, this is just a set of best practices that have been developed through engagements with customers, and it isn't a hard and fast rule that you absolutely have to use at your sites. You're going to need to analyze your specific conditions, your specific site, and then determine whether or not the documented procedures are absolutely correct for you. Maybe a task that's listed here as a daily task is something in your environment that you can get by performing weekly. Maybe one of the weekly tasks we'll talk about next is something that you need to perform daily, in your environment.

Just remember that these are best practices and what's been documented. That doesn't mean it's something that absolutely has to be done according to that documentation. That's where Microsoft Consulting Services will add tremendous value for you, because they can help use these procedures that they've implemented at other customer sites, analyze your environment, and determine what will be best for you. You may have the appropriate skill set to perform that analysis on your own, and that's quite all right. This is just one avenue for you.

Next, we'll look at the weekly tasks (slide 10). What are some of the weekly tasks that need to be performed, or are suggested to be performed in different environments, to ensure SMS operational capabilities perform efficiently?

Some of the sample tasks on the slide are monitor your site server system directories for backlogs. We talked about this, again, in one of those WebCasts I did at the every beginning of the year 2000. I mentioned performing system help checks, I think that's what I called it. I had something like 25 different directories for you to monitor on your site servers. Going through the SMS Operational Procedures document, I think I counted 20, 21, or 22 different folders to be looking at. But there's a certain set of folders or inboxes on your site server that you want to go through and you make sure you're monitoring. What you're doing is looking at those inboxes for backlogs.

For example, looking at Ddm.box, that's where all of your discovery records are going to wind up on your site server to be processed into the database. If you find a backlog of DDR files sitting there being processed, you need to analyze if that backlog is consistent, or it's standard or normal for your environment, depending upon what discovery methods you have employed; look at how often people are logging off and logging back on; look at how often you're doing Network Discovery; and look at how wide your Network Discovery's scopes are, et cetera.

You may on a normal basis have a backlog at 50 to 100 different DDRs sitting there waiting to be processed. If that's normal for you, that's fine. Just monitor that inbox to validate that you don't ever exceed 200 or 300 DDRs at a point in time. If so, if you see a few hundred DDRs, that would indicate a backlog, and you have to determine why is that backlog there. Is there some error condition that SMS is not able to recover from, so Discovery Data Manager is not able to process these DDRs quick enough? Did somebody at a child site of mine turn on Network Discovery, discover the entire enterprise, and at the child site they're flooding my site with the DDRs that they discovered?

It may be that one of our child sites is not currently running Service Pack 2, so if they're using Windows^® Networking Logon Discovery, and in releases prior to SMS 2.0 Service Pack 2, we sent all DDRs to all sites. So that may be causing an issue. But you have to analyze why the backlog is there, when normally you don't experience a backlog. So it goes through the different inboxes on your site server to monitor and look for backlogs.

The next thing you would look at is monitoring your client access point system directories for backlogs. There are five different directories on your CAP, where clients write data files to. If those files are backing up on your client access point, that could indicate that there's a connection problem between that CAP and the site server — whether it's a physical network connection problem, like the link is down, or maybe it's a security problem — the access account's been changed, and the CAP can't get a valid account to talk to the site server to move these data files over. Maybe the SMS Executive or the inbox manager assistant thread of the executive CAP has died for some reason, and it needs to be restarted. Whatever the case is, you normally don't find a lot of files backed up on client access points. But you look in the five directories there for DDRs, CCRs, hardware inventory, software inventory, and status messages, looking for backlogs. If you find some, you need to do an analysis to determine what's normal, what's not normal, and correct any abnormalities.

Attend the Change Management Review Board. We talk about, in this Operational Procedures document, setting up a Change Management Review Board, one of your teams that we'll talk about a little bit later on, to implement changes in your environment. Weekly, you'll probably want to go out there and review the changes that have been implemented, and also discuss potential changes that are being requested from your infrastructure development team, looking at whether or not that's something you really want to implement at your site. If so, what are the ramifications? What are the risks of that implementation, of adopting that change? Then meet after the change has been implemented to see what the effect is. Has it accomplished the task it was supposed to accomplish? Has it improved the performance? Has it improved the connectivity? Has it improved the distribution, whatever the case is. So attend the Change Management Review Board.

Produce management reports. Management obviously will want reports on how things are flowing, what's happening in your environment, what is a success, what's my return on investment, and so on. So you need to produce reports for your management environment.

There are a total of eight different tasks listed in the document. A couple of others would be check and improve your SQL Server performance, and manage your file system — so, looking for backlogs, looking for outdated files that are no longer needed, and removing them from your site server or other site systems. Remove any duplicate ID data, or remove any backlogs of bad MIF files that have not been processed by Inventory Data Loader.

Again, these are just sample tasks. There are more tasks in the document. And some of these tasks are things you may determine that you want to do on a daily basis, or maybe on a monthly basis, or maybe ad hoc. Again, this is just a set of best practices, things that you want to consider in your environment to try and improve your SMS procedures.

The next thing we'll talk about is monthly maintenance tasks (slide 11). Again, you have the same note. There are a few things introduced here, in the document or in the SMS Administrator's Guide, that may not necessarily be things that you want to do on a monthly basis. You may want to run them on a weekly basis, or maybe ad hoc.

Here are some of the sample monthly maintenance tasks. Compile monthly status reports — so, what's happened over the last month. What were the problems that you had? What were the problems that were recovered from or worked around? How are things working now?

Review your system health and performance. The document in the Operational Procedures has about a dozen different Performance Monitor counters that you want to look at, some OS-related, some specifically SMS related, to monitor your SMS environment. Again, look at performance, making sure that your server is maintaining a consistent level of performance, or if performance is dropping, that it's performance degradation you're expecting, because of the fact that you're adding more clients to your sites. You're expecting your site server CPU utilization to increase, or you're expecting the disk queue length to increase, because of the fact that you have more activity on your site. If that's expected, then great. If it's not expected, however, then go through and do some analysis to find out why is performance being downgraded, or why you're seeing degradation.

Maybe you want to use tools other than just PerfMon. Maybe you want to use an operational management product like Microsoft Operations Manager for monitoring your environment, or some other operational manager product — that's fine.

You want to verify or secure your SMS accounts. So make sure that none of the SMS accounts have been compromised. Make sure that if you can change the password to that account, that you're properly changing your passwords with whatever your password expiration policies are, and just verify that your security is set the way you expect it to be set, by viewing the SMS security document as part of this MCS service offering, or by viewing the SMS Security Essentials document that's available from our public Web site.

You want to test and verify that your backup media can be used to recover an SMS site. You just don't want to do daily backups and then assume that those backups are working properly, and that they were backed up properly. You need to verify every once in awhile that the backup media is correct, that you are taking valid backups, and that those backups can be used to successfully recover your site. So you want to take those backups and go to a test environment, do a recovery of them, and make sure you have backed up all the appropriate data that you need to do the proper SMS recovery and repair process.

Again, there are a total of nine different monthly tasks listed in the document, such as improving system performance, making sure that you run some sort of a disk defragmenter on a monthly basis to make sure that your processes of moving inventory files, all those different client files that move around — inventory files, status messages, DDRs, package data, and so on — aren't causing fragmentation on your server's hard drive, or downgrading or causing degradation in its performance.

The next thing we'll talk is the ad-hoc maintenance tasks (slide 12), those tasks that may not need to be scheduled on a consistent basis, but that you may want to or may have to utilize or perform on an as-needed basis.

One would be providing backup escalation and technical support. So there may be an environment or a period of time when your support staff is overwhelmed by support calls. Maybe you just implemented a service pack, or maybe you implemented some new application in your environment and it's causing a conflict, or you're trying to deploy some application through SMS and it didn't get tested out very well, so you need to go back and do some analysis. So your support staff, your help desk staff, may be overwhelmed with calls, and you, as an administrator or SMS manager, or whatever the case is, may have to help do some technical support. Or maybe there's a illness going through your environment, so you're short on staff that day. So you may have to perform some backup tasks.

You may have to distribute software packages to client workstations. You don't normally have that on a scheduled basis. It's not something like, "Oh, every Monday, we're going to generate some packages, and we'll send them out." There certainly may be some packages that you're deploying or advertising on a consistent basis, like virus scanners, disk defragmenters, and so on, but with other things, it's, "Oh, I acquired this new application. We need to deploy it now. We're implementing this new template for Microsoft Word and you need to deploy it now." So those are things that are going to come on an ad hoc or as-needed basis. So you'll be going out there and going through the process of distributing those packages, whether it's files or actual software, to your workstations or the appropriate server computers.

You have to go through and test the software packages created by your software development team. So you may have a different set of people who just create the software packages, the package definition files, the MSI files, or whatever it is. But before you implement those at your production sites, you want to validate that those packages are going to accomplish what you expect them to accomplish, that it's going to deploy properly in your environment, that they're not going to cause conflicts with any other device drivers or DLLs you already have installed, and so on. So you may have to go through the process of validating the software packages created by another group.

You may have to go through and create your own packages. So maybe you don't have a software development staff, or maybe they're too busy with some other higher-priority function, and you need to deploy this virus scanner, or a new update to it. So you have to do that on your own. You may have to go through and do those tasks.

Those are some examples of ad hoc tasks. There are 10 of them listed in the document for you. Another one would be checking the status of your security violations. SMS does a very good job of providing audit messages, stating that, "This user attempted to perform this task in the SMS Administrator Console from this workstation." You want to validate that the appropriate people are attempting to do the appropriate things in your SMS environment, and look for any audit messages, or look in your SMS Provider log file for any security violations there.

The next thing we'll talk about is the SMS operational roles (slide 13). We've talked about those different tasks that are essential to perform on some sort of a scheduled basis or ad hoc basis, and we just introduced some of them for you. Now we need to talk about the different people skills that you need to have in place to accomplish those tasks. What are the different roles or titles of employees you need to have to make sure that these tasks are implemented properly and performed adequately? So we'll look at the staffing responsibilities to ensure that your essential maintenance tasks are accomplished when they need to be accomplished.

Some of the common practices that are documented in the Operational Procedures Template document are to create three unique teams to handle your SMS operations.

The first one is the Operational Support Team. This is often the largest of the three different teams. This is the group that ensures your operational stability on a day-to-day basis. So this is your set of administrators that are doing the day-to-day administration, status verification, checking for errors, doing some troubleshooting, deploying packages, and so on. This is your day-to-day operations team. They're just ensuring the operational stability of your SMS site.

Then you have the Infrastructure Development Team. This is the team that goes through and identifies and evaluates any changes that are deemed necessary for your site. They're going to look at the architecture of your site, whether or not you need to implement a new child site because you've now just added a new WAN link to your environment, or you've added a dozen different clients at a site, and now you think that you might need to put a new child site there, as opposed to just having a remote CAP and a remote distribution point there. So they're going to go through and look at the architecture of your site. They also may be the team that's going to do your software distribution package definition files, creating your packages to be distributed through SMS.

Then they document a third team, the Change Management Team. This is the team that's going to plan and test changes to your SMS site prior to giving those changes to the Operational Support Team to deploy in your production environment. So the Change Management Team is going to take information, whether it's package definition files, packages, architectural documents, from the Infrastructure Development Team, test and verify that information, those packages, those PDFs, whatever the case may be, before they give them to the Operational Support Team. They're kind of the verification of the information provided by the Infrastructure Development Team.

Again, these are three different teams that have been identified in best practice environments as being the teams you might want to use. In your environment, you may not need all three. Or you may need more than three teams for what you need to accomplish.

The size of each team is going to vary by your environment. In your environment, you may have more packages you're deploying. You may be running inventory more frequently. You may have a larger number of clients at your site, or maybe multiple, different sites that you need to administer. It may depend on how your environment is laid out, as far as centralized or decentralized administration, we'll talk about that later on. So we can't give you, in the document, a specific number of people to perform each of these roles. You just have to have somebody, at least one person, who can perform each of these different roles to ensure that you're administering and monitoring your SMS environment in a proper fashion.

In the Operational Procedures document, there are five pages of operational roles that are documented. Now on the next slide (slide 14) that we'll look at, there's an organizational chart that will just show you the dozen different roles that are documented in that document. So we'll look at that and just introduce those roles to you very quickly. Then in the procedure document, it goes through and talks about each of those different skill sets that are required.

As we look at the organizational chart, you'll see you have an SMS Manager, and then it's broken down into the three different teams we just talked about; your Operational Support Team, your Change Management Team, and your Infrastructure Development Team, and then the appropriate roles of people to facilitate the functions of each of those different teams.

Under the Operational Support Team, you have a Troubleshooter. You have a SQL Analyst. You have a Network Analyst, an Event Screener, a Security Manager, a Site Operator, and a Backup Operator. Now whether that's one person performing all those roles, or seven different people, each performing one of those roles; whether it's four different people who are only doing the Troubleshooting role, or five different Site Operators; that's going to be dependent on your specific environment and your considerations at that environment.

Under the Change Management Team, you have a Site Installer, the person who's doing the determination of how to install your site, the site architect-type role, as well as probably performing the actual installation, because normally the Operational Support Team doesn't perform the installation of your environment. Then there's your Software Automation Test and Quality Assurance — so, testing packages and definition files developed by the Infrastructure Development Team, which has the technical architect to determine how your site should be laid out. These are software automation developers, people who are generating your package definition files, who are developing your MSI packages, and so on. Maybe you have a Network Analyst there as well, to help analyze your network infrastructure, and then determine how SMS should be deployed in your network infrastructure.

Again, the number of people in each of these different roles is going to depend on your own environment. That may be where you need a third party, as a consulting group, to come in and look at your environment, analyze it for you, and determine that for optimal operations, you might want to add another person here, or you have too many people here, whatever the case is.

Certainly, a single person may be able to perform multiple, different roles. In some small environments, it may be a single person who's accomplishing all 12 of these roles. So I'm the manager, and I also do the troubleshooting, the support, and the day-to-day operations. I do the backing up. I develop my package definition files. I'm testing everything out of my test lab myself. Most larger companies, however, are going to have multiple people who are performing those roles; maybe even multiple people performing a single role, in larger organizations. So it's again going to depend on your environment.

Now on the next slide (slide 15) we're going to do show you a sample description for one of these 12 different roles, and that happens to be for the SMS Manager. Again, there are about a dozen different roles documented in the Operational Procedures document, and what we'll do is look at one of those roles, the SMS Manager, to show you the type of information that they provide for you in the Operational Procedures document.

In this case, it's the SMS Manager, the guy at the top of the hierarchy, as far our org chart goes. So the mission for the SMS Manager would be to ensure that the SMS infrastructure meets the agreed upon service levels through the effective utilization and procedures of your SMS teams. So this person is going to make sure that SMS does what it was implemented to do.

You bought SMS to perform these tasks. The SMS Manager is going to make sure that SMS is performing those tasks optimally. So the deliverables for this person, or what role they're going to play in the environment, is they have to define and negotiate the service-level agreements with your IT staff. Your Information Technology staff is going to have specific service-level agreements that they want to have in place. This SMS Manager is going to make sure that SMS falls into that, and they know how to perform those tasks according to whatever your service level agreements are.

You're going to define the team and individual objectives. So you're going to work with each individual member of each of the different teams to ensure that they are adequately performing each of those different tasks. You can't just take this set of tasks, drop them into a bucket, and assume that, "Okay, my Operational Support Team will accomplish all of these tasks in this bucket." You have to make sure that these tasks are assigned to specific individuals so that everyone knows what their specific responsibilities are. Of course, you want to have people who are cross-trained, so that if John is not there to do my software distribution today, Jane can fill in for John and perform that task, and we're not lagging behind a day because of the fact that John got ill or had something else come up.

The SMS Manager is also working with upper management to define strategy, and to report back to upper management on the operations of your SMS environment. The knowledge or skills that the SMS Manager would have to have? Obviously, good people skills, because you're working with a number of different people, and a good understanding of the organizational mission. So why did this company purchase SMS? What were the goals of the SMS environment? What's the mission of SMS that needs to be accomplished, and how can SMS contribute to the goals of that organization?

You need to have an operational understanding of SMS, so that when members of individual teams are talking to you, saying "Well, Discovery Data Manager doesn't seem to be processing DDRs very well," you need to have an understanding of what Discovery Data Manager is, and what the significance is of the fact that someone's stating that DDM is not processing DDRs properly or efficiently. You don't have to be extremely technical. You don't have to know every single thread. You don't have to know every single gotcha that might be around or know how to troubleshoot everything. You just have to have a good overall knowledge.

So what the document is going to do for you is it's going to go through each of the dozen different roles and document this type of information for each of those different roles, as to what their deliverables are, what their mission or objectives are, and what their different skill sets are.

The third area in the document, besides essential maintenance tasks and the support roles or the people roles, are the roles and responsibilities (slide 16). In other words, tying the individual tasks to specific operational roles to ensure that each task is performed on an adequate basis. As I mentioned on the last slide, you have to make sure that you're assigning these tasks to specific people, and not just putting them into buckets, assuming somebody's going to grab it out of the bucket, take that task, and accomplished it.

You need to make sure that one person, or maybe several people, whatever the case is in your environment, knows which tasks have to be performed by them. So you need to make sure that you have key roles filled — a certain number of SMS Administrators, whether it's one person, whether it's multiple people. Again, it varies, depending on your environment. You need to make sure you have your Software Automation Developer role filled, and have your Test and QA Analyst verify the information before it gets put into your production network; have your Event Screener, the person who's monitoring the event system and status system through Windows 2000 or Windows NT^®, as well as through SMS, look for conditions that might signify an error or something that may be causing a problem, like performance: "My CPU is getting worse, so that might indicate that I need to add another CPU to my box or upgrade the CPU"; and have your operations analyst, the person who's going to be out there looking at the installation of your site or how your sites are communicating between your hierarchies, troubleshooting the different environments.

Again, as we stated before, in some environments multiple roles can be assigned to a single person. It may be multiple people performing each role. But it's key to ensure that you have adequate staffing to accomplish all the tasks, whether it's a single person or multiple people, and that those tasks are accomplished as you need them, whether it's on an as-needed basis or whether it's a scheduled basis. Again, it advantageous to cross-train employees so that you can fill in for somebody who's absent, or if you're deploying a new package, and you need to have additional help. You need to make sure you have people who are cross-trained, so that they can move from a Site Operator role into a software distribution role for a period of time to help out.

Now in the Operational Procedures document, there are 37 pages that are used to tie each of those 40 some different tasks into each of those 12 different roles. The document talks about each of the different roles and which of each of those 40 different tasks that role must perform. So it breaks out the 12-plus people into each of the different tasks that they need to perform and when they need to perform it. Is this done on a daily basis, a weekly basis, a monthly basis, whatever? It also will list the procedures for accomplishing that task, and any methods to automate that task, whether this is something that should be automated, or something that can be automated effectively or not.

Another thing that will tie into this discussion, and the thing I've said numerous times now, is that the number of people you have in these different roles is going to depend on your site. There are a lot of different factors there, and we've talked about some of those factors in previous WebCasts. But this slide talks about one of those major factors as a big decision point in your assignment of tasks and roles to specific individuals.

That big decision point or that factor is, what does your site hierarchy look like, and what is your corporate philosophy, as far as administration goes? Is your corporate philosophy that you want to have a centralized administration team? In other words, you have a single team that's performing the vast majority of your administrative tasks? Or are you in a distributed team environment role? So multiple different groups are performing some of the same tasks, but over a smaller set of SMS sites, servers, or clients.

So the more centralized you are, the less regional or distributed staff may be required. You may not have to have software distribution people in each of your different sites if you're performing centralized administration. It may require a couple of different administrators to perform software distribution to your centralized location, whatever that is. Your central location for administration will most likely be the central site. It may require more people there, but it will require fewer specialists out in the environment, spread out throughout your environment to accomplish those tasks, because one person can accomplish the same tasks for multiple, different SMS sites.

However, if your environment is a distributed environment, then you may need to have the single role employed at multiple different SMS sites, so a software distribution person at each of your SMS sites, because you're performing distributed administration. Now how many at each of those sites you need to have as a software distribution person, an event screener, or whatever the role is, depends upon your scope of the administration. Is it more of a global environment, in other words, centralized? Is it more of a regional environment, so I'm administering my specific region, which might be a dozen sites, but my entire hierarchy might be 1,000 sites? Is it more workgroup, where I'm just administering in my specific sites, my specific workgroup of team members, and then going from there and having other different people monitoring their own workgroup of team members, as far as administering SMS?

Those are things that you have to look at in your own environment. Do you need to have multiple, different people performing the same role throughout your organization, or are you looking at having a centralized administration staff, where you have fewer people performing the administration because it's going to be performed centrally?

We're down to our last few slides, and then we'll get you to the Q&A. The next thing to talk about is maintenance tools (slide 18). What are some of the tools that you're going to be using to accomplish some of these essential maintenance tasks? What tools are these different SMS roles that these people are going to be using to accomplish these tasks? Again, the document will list some of these tools. Your administrator's guide certainly covers a number of these tools for you, and there is different documentation out there on these different tools. So let's go through these very quickly. We've talked about these in different presentations in the past.

SMS system status: you have to monitor your site status, your component status, your site system status, and package and advertisement status. You have to look at status messages from your client computers to ensure the operational integrity, and make sure that there are no problems that are indicating any kind of error conditions.

SMS log files: you may not be able to rely solely on the status system to provide enough information for you to be able to operationally monitor your SMS environment. You may have to go to log files and look at those, which means you have to enable them, and then use some utility like SMS Trace to monitor those log files. One downside to the log files is that there's no great reference to our SMS Log Files. That's something I've been trying to push a lot in the SMS group, and it just hasn't happened yet. So unfortunately, the best reference I can find to the log files is some of the stuff that I've developed in some of our Microsoft Official Curriculum (MOC) training courses on SMS. Particularly, the 828 course has a lot of good information on the log files, showing you how your log files should look when things are operating properly.

Check your Windows event logs, whether it's Windows NT or Windows 2000, looking at the OS errors related to SMS, whether it's disk space or other related services.

Performance Monitor: monitoring the performance of the computers itself — the CPU, the network subsystem, the disk subsystem, memory utilization, and SMS, the specific SMS counters are available.

Use Network Monitor to check connection issues you might be seeing, whether it's access issues or network down issues, whatever the case is.

Network Trace is a good tool to look at for checking your SMS environment, for checking your SMS site system connectivity, and viewing your intranet. We've had a lot of customers who have learned a lot about their intranet by doing a network trace after having done Network Discovery and finding all the routers and subnets, and they didn't realize how many routers and subnets they had in their environment.

Use Remote Control to do testing, diagnostics, as well as control over remote systems. So for your help desk staff, instead of deploying them out to individual desktops, you may be able to do some of that support work from their own desktop using the remote tools.

Use Wbemtest or Cim Studio to utilize the ability of verifying whether or not SMS is having a problem or SQL Server is having a problem, whether it's an SMS Administrator Console issue or a SQL Server issue. Those are a couple of good utilities to use.

Microsoft Operations Manager, or if you're not using MOM, Healthmon, which comes with SMS. Utilize one of those as an operational management product to have a high-level overview of the SMS operations and your computer operations. Look for performance issues, status message issues, or just other error conditions.

You may be required to use some other resource kit utilities, whether it's the SMS resource kit or the Windows NT or Windows 2000 Resource Kit utilities, to help diagnose and solve problems as well.

A couple of additional resources for you to go to (slide 19). Obviously, the Microsoft SMS Web site at http://www.microsoft.com/smsmgmt/. You're going there for announcements about future products, whether it's SMS 2.0 Service Pack 4; information about Topaz; upgrades, like service packs; future versions, like Topaz; or updated white papers or announcements of current support. We just came up with our announcement on a support statement for Windows XP Professional clients with SMS, and that's up on the SMS Web site now.

If you want to find out more information on the Microsoft MOF, or Microsoft Operations Framework, you can go to the MOF Web site, which is at http://www.microsoft.com/business/services/mcsmof.asp, and that's where you'll find a bunch of information about the Microsoft Operations Framework.

Obviously, Microsoft Product Support Services. I'm sure the majority of you have contracts with PSS, so that you're getting great support through them.

You also have the ability to use the Knowledge Base, whether it's just up on the http://support.microsoft.com/ Web site, or whether it's through TechNet, or other resources for looking at KB articles, Knowledge Base articles related to SMS, or other things that may cause problems for SMS, those being the operating system or other services.

See your SMS documentation, that is the SMS Administrator's Guide, Chapter 18, specifically for this topic. The SMS Resource Kit, in Chapter 2, has some information on design information for your sites, and in Chapter 26, there's some information for looking at status messages, deciphering status messages, and documenting all the different status messages.

Of course, the WebCasts are all archived. You can go up to the http://support.microsoft.com/webcasts/ Web site to find past WebCasts on Windows 2000, Active Directory™, other technologies, as well as all the SMS WebCasts we've done in the past.

A summary, and then we'll kick it back over to Jason. We talked about SMS Operational Procedures. The key here is that the more time you spend planning your ongoing maintenance and performing those ongoing maintenance tasks, the more successful your SMS environment will be. The less time you spend planning prior to deployment, or, now that you're already deployed, the less time you spend performing maintenance procedures and maintenance tasks for your environment, the less successful your environment will be over the long term.

The goal of this presentation, as well as the Microsoft Operational Procedures Template document, is to give you procedures to get you best practices so that your SMS environment can be running optimally on an ongoing basis, and if you do have down periods, get them back up and running as soon as possible with a minimum amount of time lost. So in this presentation we talked about SMS maintenance tasks, staffing roles, procedures for maintaining SMS, the tasks that you want to perform daily, weekly, monthly, and on an ad hoc basis, and we talked about a dozen different staffing roles to perform those tasks.

Again, if you specifically want the SMS Operational Procedures Template, that document I've been referring to, or some of the other documents I introduced earlier, you get that through engaging Microsoft Consulting Services, or a third-party consulting service that has been trained on this, to engage in and acquire the SMS service offering.

At that, we'll kick it over to Jason and let Jason kick off the Q&A session.

Jason Bennet: Thank you for that presentation, Wally. Just a couple of quick notes before we move on to the Q&A portion of the Support WebCast. To access information on all upcoming Support WebCasts and the archived information from all past WebCasts, an easy-to-remember URL is http://support.microsoft.com/webcasts/.

I did want to make another quick comment. We do have a lot of one-on-one support questions coming in. Unfortunately, we just don't have the time to address all of those in this WebCast. Typically, when I talk about one-on-one support, I mean if you have a particular configuration or you have a particular setup, and you need some advice on setting that up, you're really going to want to talk to Product Support Services. You're really going to want to call in to them and speak to a Support Professional. It's just more out of the range of the discussion than what we're doing here today.

So with all of those caveats out of the way, let's get started. The first question I have is: Is there information about planning system changes after a significant change to site infrastructure; for example, moving from a single site to a multisite environment?

Wally: Information about planning system changes from a single site to multiple site environments. There's some of that information in the SMS 2.0 Resource Guide. Chapter 2 has some design information. In your administrator's guide, there's some information on planning your hierarchy. Now as far moving from a single site to a multiple site environment, I haven't seen any document that says, "When you go from a single site to a multiple site environment, here are the considerations you need to look at." But primarily it's the same information you've been using, the same procedures in a single site environment. You're doing the same things.

What's been changed, however, is that instead of maybe 10,000 clients that you're administering in your single site, you might now have three different sites, ranging from to 2,000 to 3,000 to 4,000 and maybe 5,000 clients each. So the information, the amount of administration, on a per-site basis, may very well be less that you have to look at, feed through, analyze status for, and error conditions for, and so on.

The other area that you have to be looking at is whether or not you want to go with a hierarchical environment, having a central site that receives all the information from all of your child sites, versus having a separate SMS site totally. Most likely, in a single company, you're going to want to have a hierarchical environment so you have a central site, and all the information is set up to a central location for complete analysis, for complete asset tracking, for complete inventory, and so on.

The other big area, when you go from a single site to multiple sites, is your communications between those sites. So you need to make sure that you understand what the communications are between one SMS site and another SMS site. What information flows down the SMS hierarchy, and what information flows up the SMS hierarchy. What information can you control in that flow, and how can you, or can you at all, control the bandwidth utilization of SMS between those two different sites? SMS does have very good functions and capabilities for controlling your bandwidth between SMS sites. So that's a definite possibility, but you're looking at connection accounts there — the amount of traffic that's flowing between, when information is flowing — the fact that now this information that used to be more readily available to me may now be delayed because of the fact that I'm in hierarchical environment, and it takes time for that information to flow up the hierarchy.

As far as specific documentation on moving from one to another, unless there's something in the admin guide, and I think it's Chapter 3, on planning your environment, or the Resource Kit in Chapter 2, the design information, I haven't seen much that I can think of.

Jason: We have a couple of questions coming in about the location of the Operational Template. Several people have been asking where they can get it, and is it currently available. One person asked: Is the template available to me if we purchase Premier Support?

Wally: When I checked with people on this, when I was asked to do this presentation, I was told that currently the only way to get the Microsoft Operational Procedures Template is through MCS. So it is an MCS product offering, and I tried not to make this a sales pitch for MCS, although it may have sounded like it, and I apologize. That was not my goal at all. It was to try to give you some information, and then let you know that there is further information available. However, this specific document and the other things, that MCS service offering, are currently only available through MCS or other consulting corporations that have been trained for this specific process, the SMS service offering.

However, with that said, a lot of the information, or a lot of the day-to-day and what we call the essential maintenance tasks, a lot of those are documented in Chapter 18 of the SMS Administrator's Guide. What's not documented anywhere I can find in the SMS documentation, and that is only available in this Operational Procedures document through MCS, are the staffing roles. The dozen different staffing roles, that is, the org chart we saw here, and the tasks each of those different roles have performed. That I have not seen anywhere, other than this document.

With that said, I know in the future, with Topaz (I got hold of some the chapters of the Topaz administrator's guide and procedural documents) they are starting to put some more of that information in there. They're talking about different types of staffing roles and the essential maintenance tasks. I don't know if they're going to be as complete as the MCS product is, but they are making some progress there.

As of right now, I'm not aware of it being available through PSS. It's only available through MCS or another third-party consulting service.

Jason: That seems to tie into a lot of the remaining questions we have, which regard specific recommendations for staffing for different sized organizations. Can you speak to that? Do you have any recommendations as to the number of people needed?

Wally: Like I said throughout the presentation, there's no one answer for any company as far as how many SMS staffers do you need to accomplish and to perform your operational procedures effectively. There just isn't an answer. The best rule of thumb answer we have, which is kind of the generic guideline answer, and I think I mentioned this in one of my WebCasts almost two years ago, is to look at the number of administrators you have for another complex product, like an Exchange, something like that. Make sure that you have at least as many SMS administrators as you would have Exchange administrators, because SMS is more complex product than Exchange. It touches more of your infrastructure than Exchange does. And I don't mean to offend anybody who thinks Exchange is complex, but in most peoples' opinion, SMS is the most complex product that Microsoft has. So you need to make sure you have at least as many administrators as you have for your Exchange environment, because theoretically, SMS is going to be more complex. But for specific environments and specific scenarios, there isn't a single common answer for that.

That's where you may need the analysis of a third party, whether it's a third party within your organization, somebody outside the SMS environment, to look at your procedures, what you have now, and the procedures for what you need to accomplish, the people you have, the different staffers that you already have assigned, and the tasks you want to accomplish. With your IT staff, and knowing what kind of procedures they had to employ for supporting other products, maybe it is an MCS engagement. Maybe it is whatever other consulting product you have, or process you have. Premier Support has supportability reviews. They have those reviews that they'll perform for your site environment, design reviews or supportability reviews. They can help tell you whether or not, in their experience with working with other customers, you need to have more people performing this role, or fewer people.

Unfortunately, there's not a single answer that will work for everybody. But again, the general guideline is at least as many administrators as you have for Exchange.

Jason: You mentioned Topaz a minute or two ago, and I don't know if this on topic, but someone wanted to know a little bit more about the migration to Topaz: Will there be a logical migration process from SMS 2.0 Service Pack 3 to Topaz, and how difficult does it look?

Wally: Sure. Unless there are requests for different WebCasts, what I'm going to start doing next month is delving into specific areas of Topaz. Last month, we did the overview of Topaz, where I kind of hit at a high level each of the different features, the new things in Topaz. So what I'm going to do next month, on Thursday, November 29, the Thursday after Thanksgiving, is "New Microsoft Systems Management Server (code named Topaz) and Active Directory Integration" (http://support.microsoft.com/servicedesks/webcasts/wc112901/wcblurb112901.asp). So what I'm going to start doing next month, and then as long as we need to for Topaz, is delve more deeply into specific areas of Topaz to get you more information on it. The next one is Active Directory integration.

Specifically for an upgrade, yes, there will be an upgrade from SMS 2.0 to Topaz. It will require a base of SMS 2.0 Service Pack 2. So we will not support an upgrade from Service Pack 1 or RTM code, nor from SMS 1.2. We're upgrading from SMS 2.0 Service Pack 2 or later. It could be Service Pack 3 or Service Pack 4 when it comes out, up to Topaz.

The upgrade will be like a service pack. So when you get a service pack, you download the service pack from the Web — although you're not going to download Topaz from the Web — but you get a CD. Either you download it from the Web or you get a CD that you order from the Web site. In this case, you'll order a Topaz upgrade kit. You'll put the Topaz CD in. You'll run Setup. There will be a readiness analyzer in the upgrade process to analyze your existing environment, telling you whether or not you can upgrade your existing site to Topaz. And if not, why you can't. "Well, it's because I have NetWare out there, and Topaz is not going to support NetWare site systems."

Actually, Topaz will take care some of that automatically. It will deinstall your NetWare site systems. But it will give you, through that readiness analyzer, a picture of what's going to happen to your environment during the upgrade from SMS 2.0 Service Pack 2 or later, up to Topaz, and things that it can't do. For example, you still have an SMS 1.2 child site, and we don't like that. So you'll have that, but you're going to run the Topaz CD. You run Setup off of the CD, and it's going to go through, do the analysis for you, or run the readiness analyzer prior to trying to do your upgrade.

Then it will go through an upgrade of your site from SMS 2.0 up to Topaz, very much like a service pack. You punch it in. It does the work for you. We don't know yet whether it's going to have a reboot, mostly likely not. Hopefully, it won't require a reboot. Then your site comes up as a Topaz site. Then it pushes out the appropriate Topaz bits off to all your other site systems that are still applicable. Then you have to go through and add whatever site systems you need, being management points, server locator points, reporting points, or whatever the case is in Topaz. We'll get to all those in future presentations.

But yes, there will be an upgrade from SMS 2.0 Service Pack 2 or later to Topaz. I haven't tried it yet, but it's supposed to be fairly seamless, just like a service pack now, for SMS 2.0.

Jason: At my company, we're going to use SMS to push an advertisement for an anti-virus software. We have an issue with users deleting the software after it's installed. Is there any way to automate the process of screening for what systems don't have it installed and repush the advertisements?

Wally: This is the old "users getting in there and monkeying with your software distribution environment" problem. The best way of preventing that is, actually it kind of leads into something else I wanted to mention. Microsoft just came out with the Microsoft Security Tool Kit. It was just released a couple of weeks ago, and it's in response to all of the virus attacks that we've had on IIS and some of the other technologies. In that set of procedures, all those patches that were developed to be deployed, we worked really hard in the SMS group to develop a set of procedures and documentation for deploying SMS out into your environment, and using SMS to deploy these security patches in your environment. So there's some great documentation. There's a set of collections and queries to be imported, package definition files, and so on, to help SMS deploy these security patches.

As part of that process, we had to analyze your existing systems to determine whether or not they had a secure version of Internet Explorer, or what version of IIS was running, or what version of Terminal Services and service packs were running, and so on. So what we did was we wrote SMS Installer utilities, or wrappers, that would query the registry of your computer, using SMS Installer, looking for some sort of a registry key. We pulled back the version of that registry. So if we go out there and find Internet Explorer, we pulled back the version of Internet Explorer from the registry. Then we used that to determine whether or not we need to apply a security patch to Internet Explorer.

You can certainly do the same thing here for your environment. After you've deployed your virus scanning software, you want to verify that the users have not gone out there and removed that software. So you could run one of these SMS Installer-type scripts to scan the registry. You advertise it through SMS. It runs through the scanning process. It reports back a success or failure, however you want to have your script run, as to what computers have this software installed, or which ones don't have it installed.

Specifically, within SMS, after you deploy a product to SMS clients, it depends on what your collection criteria was. If your collection criteria was, "Show me all computers that don't have ABC Virus Scanner installed, and then if they don't have it, they fall into this collection," well, after you deploy that ABC virus scanner out there, then they'll fall out of that collection. If a user removes that product and the next time you do hardware/software inventory, whatever you're doing to determine whether or not you have that product installed, it's not there any more, it would fall back into that collection, and they would be redistributed that package automatically.

Outside that, it's a manual process of running some sort of an advertisement to go out there and scan to see if something has been removed. SMS doesn't track itself, whether or not, "Okay, SMS advertised this, and oops, now somebody's removed it." We don't track that for you, unless there's a status message query. Let me look to see if there's a status message query that would say that clients that removed an advertised program. There's a possibility of that. Other than that, SMS will do some of that automatically when your computer falls back into a collection that it shouldn't be in, because it should have already had that product installed, or you run an SMS Installer script or something else. I don't see anything, a status message query, that says a user removed an advertised product. So I don't see that.

Jason: I have a number of clients where the last run time is old, in months. How do I get these to update? Also, not all of my clients appear as a choice to report on for software metering. How do I correct this? Is this one-on-one support?

Wally: As far as the last run time is old, I don't know what a last run time is. If you mean the last time they reported discovery data, then obviously you need to get some sort of discovery method running out there, but I can't think of a last run time. So please write back and state what you're specifically talking about, for the last run time.

If it's looking at discovery data in the database in one of the collections, and you see that the agent last scan time or agent time is very old, then it's figuring out what discovery methods you have employed and what discovery methods should be deployed or utilized in your environment, and why this client has not run that discovery method. Maybe it's the fact that their client access point has been decommissioned. So they haven't updated to find new client access points. Maybe that's because they can't access, because the client connection account has become invalid, the password was changed, and they didn't retrieve the new password before the CAP went down, or whatever the case is. Maybe they haven't hit a logon point in a long period of time. So it may be just a matter of running Smsls.bat or Smsman.exe to hit a logon point and update the client to get him kicked back in and back up and running. But there you'd really have to look at the log files in your client, probably Ccim32.log in to client computer to see what it's reporting. It may very well be that there is some sort of a connection issue getting into the CAP, so it can't retrieve any settings, if that's what you're talking about.

As far as the software metering agent and clients don't appear, clients appear in your software metering products list, or as far as lists for software metering, after they have the agent installed and they start generating some data. So after clients start generating data, they'll appear in your software metering admin console, as far as workstations you can exclude or set permissions on, or valid run times on, and so on. Or you can do a scan within the admin console for that. Maybe these same clients that aren't running your run time is old, they can't connect to the CAP, so they don't know they're supposed to install software metering. So they haven't installed it. Then they're not reporting any data, or maybe they can't find a software metering server. So there you'd have to look at the Liccli.log file on the client to determine what's going on there.

Jason: Is Service Pack 4 for SMS coming soon? What will it do for us? Someone else is asking what the highlights are.

Wally: Service Pack 4, we're targeting it before the end of this year. So it should be coming out before the end of this year. Basically, it's going to be just like Service Pack 3 was, which is a rollup of fixes. So it's going to be a rollup of any of the hot fixes or QFEs that some of you might have deployed individually, when you had a problem. I'm not aware of any new features being added to Service Pack 4, other than whatever support is necessary for Windows XP, as well as whatever the current release of .NET Server is. But I'm not aware of any other advancements or any other feature changes.

After we got to Service Pack 2 to give us a stable platform for SMS, all of our service packs from now on are supposed to be just like Windows service packs, which is just a rollup of fixes, and not adding new functionality. It's just that, honestly, the RTM of SMS 2.0 was not of great quality, so when we came out with Service Pack 1, we added a lot of functionality to try to improve that. So a lot changes were made. Then Service Pack 1 had its issues. So we had to make a lot of changes for Service Pack 2. But Service Pack 2 got us to a quality bar that is good enough, it's stable enough, that everybody should be there, and everybody's platform should be stable enough, with a few exceptions. That's why we have hot fixes and come out with other service packs. But we don't have to introduce new functionality in service packs anymore. Our services packs are primarily going to be rollups of QFE fixes.

So that's all the targeting that I'm aware of for Service Pack 4. I'm not aware of any specific feature additive capabilities in Service Pack 4, other than whatever we need to do for .NET Server, or anything we come up with for Windows XP. Again, it is targeted to come out by the end of the year.

Jason: I did want to take a moment to solicit some feedback from our audience. If you've tuned in to us for the first time or you're coming in as a seasoned veteran of these Support WebCasts, if you want to give us some feedback on the interface, the presenter, the topics we've been covering, if you want to suggest new topics, you can send it to feedback@microsoft.com. Just make sure you put "Support WebCast" in the subject line.

The next question regarding the organizational template: Does the template allow modifications of the roles due to other products taking over some SMS roles, such as software deployment through group policies in Windows 2000?

Wally: I would debate the fact that roles are being taken over by software group policies in Windows 2000. It may be in your specific environment that you can accomplish all your software distribution needs through group policies and IntelliMirror^® in Windows 2000. But we've had WebCasts in the past and there's documentation, white papers up on the SMS Web site, that talk about the differences between SMS software distribution and Windows 2000 software distribution.

So my first stab at an answer, not to be critical at all, is that you may be fooling yourself to think that you can provide all your software distribution needs through Windows 2000, just because it has group policies and IntelliMirror. There's a lot of different things that SMS can do, that group policies cannot do, or IntelliMirror cannot do, for software distribution.

Now there are also some things that group policies may be able to do for you that SMS can't do. So there's a two-way street there. But again, it may very well be that you can accomplish all your needs through group policies. If that's the case, then no, the Operational Template is written specifically for SMS, and this was written almost two years ago now. So it has been out for quite awhile, just not widely known. So it has not been updated to accommodate group policies, procedures, and IntelliMirror for software distribution in lieu of SMS.

Maybe you're just referring to MSI, and using MSI packages as opposed to SMS packages. SMS can distribute MSI installer packages fine. It has great procedures for doing so, and Topaz will improve that, we'll talk about that coming up. But the Operational Procedures Template does not accommodate that. Now that doesn't mean that your third-party consulting service or MCS or whoever it is working with you with this Operational Procedures can't go in there and say, "Okay, in your environment, you are using group policies. You are using IntelliMirror, and that is accomplishing all your software distribution needs. That's great." Then they can adopt that to whatever is stated inside the document itself for that, but it does not make references to that because, again, it came out not too long after SMS 2.0 came out, which is prior to Windows 2000 coming out.

Jason: Are there any documented procedures anywhere for running DBCCs?

Wally: Yes, there are documented procedures for DBCCs. There are some in the SMS Administrator's Guide. In Bruce's presentation, August 28, 2001, he went through DBCC. Of course, the SQL Server documentation is where you're going to find the most explicit references to DBCC. Also, I'm sure there's something up on the SMS Web site. If you go to the Maintenance and Recovery location for that, and go to the Recovery Expert (under Tools) and some of the documentation for the Recovery Expert, there will be information there for performing DBCCs before you do your backups and so on.

So there are numerous different places, but I'm pretty sure we have it in Chapter 18 of the Administrator's Guide, for maintaining your database. It's the second part of Chapter 18, Maintaining SQL Server {Follow-up: The chapter is Maintaining SMS Systems, the topic is Maintaining SMS Databases}. I'm sure there's something there, as well as Bruce's presentation. I'm not aware if there's a white paper up on the Web site that would specifically talk about it. Of course, SQL Server documentation, then the Recovery Expert, and the Maintenance and Recovery information up there, would give you some documentation on that as well.

Jason: It seems to me that information on contacting Microsoft Consulting Services is a little vague. Can you give us some specifics?

Wally: Okay. Contacting MCS, you would just have to find your local Microsoft office, communicate with them, and state that you want to have some discussions with Microsoft Consulting Services for looking at whatever the appropriate technology was. If you're talking about SMS, tell them "I have an SMS environment. I want to engage MCS to come in and look at my SMS environment to see if it's running optimally; to see what procedures I need to change to make sure things are working optimally; to recommend any changes in my hierarchy, because I did this all on my own, or my consultants that I had. It was two years ago, and I want to see what's changed." You want to see if your environment is looking okay for upgrading to Topaz, as we start looking into Topaz, or maybe you want to migrate from Windows NT 4.0 up to Windows 2000 and Active Directory now. Are there any considerations there?

So really it's just a matter of contacting your local Microsoft office and getting into contact with the MCS representative there, and they'll have multiple MCS people at each site. But there is a manager of MCS or a coordinator. Tell them you need to schedule some time or set up a meeting with MCS, or however you want to go about this, to go and talk about the potential for them engaging into an environment with you to look at whatever your appropriate product is. Again, this specific product for SMS is the SMS service offering through MCS.

So you just have to contact Microsoft Consulting Services at your appropriate site. They may not have anybody at your appropriate office, at your local office, who can handle this. They may have to have somebody from a different district come in and help you out, and that's fine. They're well aware of that, and they have the procedures to get you the information that you need to make sure that SMS, or whatever product — MCS covers all of our products — is running optimally, as best as it can, so that you can utilize our product efficiently and maximize that return on investment.

Jason: This question is a follow up to the question on software metering and client run time, and here's the follow up: When I choose Software metering from the Tools option on the console, I then choose Last client run time, that is when my last client run time shows as old. Did you say I should consider my discovery methods? I'm using Windows Networking Logon Discovery. The CAPs have not changed.

Wally: So the discovery methods — the reference there was because I wasn't aware of what your last run time was about. If it's specifically about software metering, then you don't have to worry about your discovery methods. So that was just thinking that you might have been referring to your last scan time for your discovery process. So the CAPs and the Windows Networking Logon Discovery are out of the picture.

So, looking at your software metering and from the Tools console, the Last run time, why your clients are not updating. You would have to go look at your individual clients that are not updating and look at the Liccli.log, the one I mentioned before, and see what it says. It may be that, if you look in there, it's going to give you an error 2, Cannot find a server, indicating it can't find a software metering server. So maybe there's an access account issue where it's not able to access a local software metering server. Maybe it hasn't updated this list of software metering servers in a while, which may mean that it can't access the CAP. So you might want to look again at the Ccim32.log to see if it is updating. You can go look at your client registry, and I don't have it off the top of my head, because I don't have a software metering client, but it's is HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SMS\Client\Sites\System\sitecode\Client Components\License Metering and the License Servers key. In the License Servers key, it should list for you all the different software metering servers you have available. Make sure that that list is valid and that it has up-to-date servers. Maybe for whatever reason you've decommissioned some software metering servers, they're not there anymore, and put up some new ones or changed the names, and for whatever reason some of these clients aren't updating.

From there, you have to look at the log files. You have to analyze whether the client's talking to CAPs or anything else, and doing its discovery data. So in other words, it's reporting information properly, then why it can't talk to software metering? Maybe that client has been excluded from running programs, or from there it's getting into individual product support. You need to look at those log files, and you need to go to PSS if you can't find anything out. I can't tell you much more than that, unfortunately.

Jason: To test the restore of this SMS database, what exactly are your recommendations? I'm talking about testing connectivity, since I assume that if you restore the SMS box and are able to do queries, that would assure database connectivity.

Wally: There's a lot more than just restoring your SMS database in the recovery process. If you think an SMS restore is just the process of restoring the database from SQL Server, most likely you could be causing corruption in your SMS environment, because there is information in the registry that has to be tied to the SQL Server database, that's also tied to files in your file structure. So any time you do a restore of an SMS site server, you need to make sure that you restore not only the SQL, but also the registry information, as well as the file structure. So there are a lot of things that are tied together with that.

So when I talk about testing a restore process, take your backup process. So in your Site Hierarchy, <Site Code>, Site Settings, Database Maintenance, then Tasks, there is a task called Back up SMS Site Server. I'm assuming that's the task you're running. If not, that is a task that you need to be running to back up your site server. That task will successfully back up your registry, your file structure, and your SQL Server database. It will produce some specific configuration information about your computers — the SQL Server computer, your site server and so on — so that if you do need to recover that, you have all the information you need.

So you take that backup process, you take all that information, dump it to some sort of backup media, Jaz, ZIP, CD-ROM, whatever your method is, take that to your test lab where you have the exact same computer names, the exact same scenarios set up, just in a smaller environment. You then restore that backup — the registry, the file structure, and the SQL Server database. You go in there, after you've done that restore, you bring your site server back up. Now you're testing your queries to see if you're pulling data. You're testing your clients to see if your test lab can talk to your site servers, the client access points, and so on. So you're testing your procedures that way to validate your procedures. You're validating your backup media has been successfully generated, and that you know how to do your recovery process.

We did a session, I believe it was back in June 2000, let's go back and look. But go back to the archives of the WebCast, and there was a session that we did not terribly long ago on backup and recovery (http://support.microsoft.com/servicedesks/webcasts/wc070600/wcblurb070600.asp). It talked about how you do a backup and recovery utilizing the SMS failure and recovery Web site up on http://www.microsoft.com/smsmgmt/techdetails/recovery/ to document how to do a proper backup, and then procedures for doing a recovery that's goes through step-by-step what you have to do for your recovery and repair processes, especially in a site hierarchy environment. So I'd highly recommend that you go look at the recovery process up there, so at http://www.microsoft.com/smsmgmt/techdetails/recovery/, and then go look at the Recovery Expert. There's a tool up there called the Recovery Expert that will document all the different steps you need to go through to properly back up and then restore your SMS site.

Jason: I did want to comment on some of Wally's comments mentioning past WebCasts. You can, of course, go to today's page and look at the WebCasts for the next month, and that's at http://support.microsoft.com/webcasts/. There's no www on that. If you look down at the bottom of that page, you'll see a link to the Past Support Webcasts, and that will open up another page that has a tree structure, grouped by product. So if you want to look at Bruce Jones' WebCast from August, the easiest way to do that is to click on the Systems Management product tree and click on the one for that date. That will take you to a page that contains the PowerPoint^® files and the streaming media with the PowerPoint files embedded. It also has the transcript, and the transcript will be posted to these sites three weeks after. So these are some great ways to go back and look at old WebCasts, because a lot of this stuff is still very, very cogent to what we're doing now.

Wally: Just for reference, Jason said three weeks, I heard the last one I did it took longer than that, whether they were backlogged because of the number of WebCasts or whatever. Don't be surprised, or don't be shocked or don't e-mail me and say, "Wally, where's your transcript?" if it's not up there in three weeks, because the last one I heard was longer.

Jason: Yes, it's a rule of thumb, it's about three weeks, but you should definitely send us feedback to feedback@microsoft.com if you're not seeing that WebCast transcript posted up there. We're tracking it, and we're trying to make sure it gets up there, but if you don't see something, you can always write in, and include "Support WebCast" in the subject line.

That does finish all our questions, so we're going to wrap up our session. I want to thank everyone for joining us, and I hope the information was useful today. Again, we are very interested in your feedback. You can send it to feedback@microsoft.com, and include "Support WebCast" in the subject line. We hope you join us again in the near future. Thank you, and good-bye.