SAP Data Migration

As featured on Data Migration Pro “Ask an Expert”

“I’m going on my 1st migration project: we are migrating data from a legacy system to SAP. What are some of the fundamentals that I need in preparing the data in the legacy system?”

From the nature of your question I am going to assume that you are one of the legacy experts co-opted onto the project and take it from there.

Firstly, it is important to realise what SAP is and how it works. At its heart SAP is not only a computer system but a set of business processes: A theory about how stock in warehouses should best be managed, how invoices should be written, how stock should best be purchased, how late accounts should be chased and so on. This redefines people’s jobs in the post go live organisation around the “new ways of working” and with this they will also receive a computer system and screens which embodies this – SAP.

So one thing you should be prepared for is the project environment itself. A major feature from the outset will be the Process Team. These are business analysts who will select the processes and design the screens. One of their major aims will be to interconnect the way the business works in a manner that may not have been possible before. So that for instance agreeing a sale with a customer will create a raw materials request and a production plan requirement immediately and those people will be able to see the sale go through and check its details themselves even though they are not in the sales department. This creates a lot of new joined up thinking across the company, increases its reaction times and hence decreases stock that needs to be held on a “just in case” basis. Returning to the data (in case you thought I had forgotten it!) joining up the business logic in new ways will require the data to be joined up in new ways.

So that in summary key things you need to remember are:

– People will not work the same way as they did before
– The supporting data will not exactly look the same as it did in legacy
– Data that was on different systems before may have to join perfectly for the SAP load

This last point can create great strain on the data preparation as while Referential Integrity is often good within systems it is not always good between systems.

The first two points mean that you cannot assume that data is required the same way in SAP as it is in legacy or even needs to work in the same processes as it did in legacy. In fact it would be easy to over prepare by making too many assumptions about how SAP works before this is revealed within the project lifecycle.

I would suggest that you:

1) Identify all the entities in the organisation, draw a big map of which are on which systems and which are the master systems in each case. Note particularly if you have two islands of information for the same object. E.g. Customer information behind the web site and also customer information on a telephone orders system. These may well have to be consolidated and de-duplicated for SAP into a single list.

2) When you are completely sure you have an enterprise wide view of what data is where start profiling the main entities using a data profiling tool. By creating profiles and storing them you are creating a resource you can return to when questions are asked later by the process team or the SAP upload team.

3) Locate all the manuals that say how the data works in legacy and what they support. Look especially for documentation of tweaks if the legacy system itself was a standardised package that has been altered or customised.

4) Return to your data map and try to put names of people in the organisation to each entity. Where do these things come from? Who really owns them or knows the meaning of the data well?

5) Create some simple “whole of entity” extracts. Prepare them in Access or Excel ready for the process team or the SAP upload team to view.

By the end of this process you should be able to demonstrate where things like Customer, Supplier, Chart of Accounts and Material Master lie on legacy, how large they are in terms of rows and what lies within them.

Only when you have these information sources to hand would I consider actually cleansing anything. And if you do clean anything go for things that you are really very sure will be a feature of the SAP system. It is a mistake to try and clean the whole of legacy because some of it will be relevant to old ways of working and not to the new – meaning it will get left behind!

In closing – I have said overall:

– Expect SAP to be different
– Prepare mostly by gathering information about data, not by adjusting data
– Don’t make too many advance assumptions about the target

If you achieve even part of the above you will find the incoming SAP experts incredibly pleased and surprised with what the base they have to work from.

As featured on Data Migration Pro “Ask an Expert”

We will soon be migrating from JDE to SAP.   How should we proceed?   The question where we perceive complexity is with Media Objects.   Can you offer some advice?

A Media Object is a file or text attached to a JD Edwards record. Media objects can be text, images, files or shortcuts. (Ref)

So that, as the questioner already knows, it is a very “mixed bag” of data. I would agree with him, that in technical terms this is likely to be a large sticking point.

Firstly he should look at making the problem just go away. I would suggest talking to the Programme Manager and the process team and highlight the attachments as a considerable challenge on top of that of configuring and running the SAP system. He/She will hopefully engage with the issue, look at what the attachments are in business terms and why history is required from a new process design perspective. In 4 out of 5 projects luck will be with the questioner and the Programme Manager will take the risk equation in hand and just declare the legacy attachments out of bounds for migration, backed up by a moth-balled JDE implementation if they do ever need to be accessed. So that the JDE record itself is migrated, if even that level of history is required on SAP, and potentially a single attachment is created in all cases where one or more previous items existed with a standard text such as, “This historical record formerly had attachments that may still be viewed on the frozen JDE instance”.

The main thing is to take the question outside the technical team and treat it as a project risk rather than something he has to hold onto and solve personally. My own experience is that SAP migrations are risky, costly and driven by very specific aims such as better control of finances. The delicate cost, benefit and risk balance of the migration often dictates that items such as attachments must be left behind along with very large volumes of history, because they are not directly associated with the sought business benefit or future operations, but are just an archive. For example, it is common practice to migrate open purchase orders only, and leave those already filled behind. This is often allied with deliberate business tactics to drive open business down such as pre-ordering and filling the warehouse.

But let us say that the questioner is unfortunate in this case, or manages to minimise the number of such records significantly as above, but is still challenged to move some of the attachment data.

The key questions in terms of SAP migration would appear to be:

1) How do we get the attachments out?
2) How and where do we put them back into SAP?

I feel what he should be looking for is some kind of JDE reporting facility, perhaps bringing alongside something like Oracle BI Publisher (which integrates well as Oracle own JDE) Then he can use the notion of “publishing” the Media Object attachment to a relatively neutral format such as PDF rather than trying to migrate or convert it at the data level. The reload task is then one of re-attachment, not of worrying about BLOB (large binary object) fields, their internal document formats and the like.

If he really does need to put his hand “in amongst the cogs”, which I do strongly suggest you avoid, a third party tool may be of use in unpicking the JDE data structures. There appear to be several here at EverestSoft.  (Note that this is not a recommendation. I have not used any of these tools myself)

But really – why would he do that? The Objects are attached “documents” to start with, so publish and reload makes complete sense, if indeed he has to migrate this data at all.

We are working on a large project and as it has progressed I have become increasingly worried about the progress made. The legacy systems seem to be less well maintained and more complex in their interconnections than the client had said and things do not seem to move very far each week. We are contracted to do the in flight cleanse, transformation and upload but not the extraction.What do I do about the lack of co-operation from the customer side?

One simple answer to this complex problem is that your client appears underpowered in terms of large scale migration experience on their side of the contractual responsibility, but they cannot ask you for that help without further muddying responsibilities. They have agreed to share the problem but have no real way in which to deliver on that promise. What they need right now is their own migration guru, so that the joint responsibility in the contract is backed by joint knowledge and actual capability to act. Because this has not been happening so far you gave felt compelled to reach across the contractual divide to their side in an attempt to de-risk. In fact this just blurs the contract further and could ultimately place you in danger of looking responsible for the client’s shortcomings precisely because you tried to be helpful on things that were really for them to deal with. I think it will probably be better for you to keep a firm divide and invite them to “Bolster the Home Team” as described on our home page for both your benefits.

If responsibilities for action do become unclear and the project subsequently fails the client is likely to cause you upset with bad publicity and of course the personal professional disappointment of project failure. Equally, they are very unlikely to make a legal case for compensation stick because of the ambiguity or receive a delivered system either. This really is an “everyone loses” scenario that both sides would do well to avoid.

If you have been moved to write as an implementation partner the client is probably even more concerned at this point. A brief conversation with an independent data migration consultant, such as myself, may be enough to convince them to hire the help they need and minimise both their risks and yours at the same time.