(solved) Who can really support about meta-data making? = yes forum users and myself

kitakoredaz

after all, daz can not clean up their product meta-data,,  how many years they used,,, 

Jaderail

Because the Tech Support staff does not write Metadata files or know the answer to the question you asked. They are Tech as in why it runs and how it runs, The program and its support programs. The People that create the Metadata files are in a different department of DAZ 3D and as far as I know do not answer support tickets or even see them. I have no idea who a user should ask such a question too.

kitakoredaz

good night,, jaderraitl,,, I miss you actually often. 

sioc

I understand you prefered to have a daz answer...

There is that http://docs.daz3d.com/doku.php/public/software/dazstudio/4/userguide/creating_content/packaging/tutorials/adding_metadata/start

I think there is one or two thread in forums here about meta data, and you can find how-to on cgshare and deviantart.

Jaderail

I have never written Metadata I have never seen a Content DB Editor, I have saved my custom Categories and used Export User Data from the Content DB Maintenance menu. I have not seen a Metadata Update change any of my saved categories, but I will admit I have less than 1/4th of my installed content categorized at this time.
I still say only a person that does the Metadata or the update files for items could answer your question. It is not a tech issue it is a file issue they would not know because they do not write hand made Metadata that might be replaced. They work for DAZ 3D and only see the finished metadata in use. There would not be a reason for anyone at DAZ 3D to write metadata to be used that way. They would only write metadata to be released to us Users as a update to a product that had none. I can not see anyone at DAZ 3D having hand written metadata on there PC's to test with.
This is what I mean. All metadata at DAZ 3D is full metadata no User made data to test.

kitakoredaz

Jaderail

Here once again I say this. A Bug report on a product goes to the department that can correct the error, they get sent to the proper department so errors that effect all users can be fixed. A Tech support ticket asking how Metadata works for user made Metadata is not a Tech issue and I again point out I do not know who such a request should go to. As the Tech Deptarment did not answer your question it is clear to me it is not a question the Tech department can answer. That is all I can say on the subject as it is all I know.

kitakoredaz

you have been real good man,,I do not forget of you. 

Jaderail

I'm not defending anyone Kit I'm simply posting what I believe. Just as you have. I do not see it as a Tech issue but a issue for another department whoever that may be, and you see it as one the Tech department should handle.
I agree we disagree on our view of this issue. If I could help I would.

kitakoredaz

Wilmap

Haslor is the man to ask. He knows all about metadata. He showed me how to do it.

There is this thread here, which might help, but you could send him a private message and ask for his help.

http://www.daz3d.com/forums/discussion/24231/

kitakoredaz

Thanks willmap,, I like many your items which you offered as free. 

Wilmap

Unfortunately I haven't amended any other products, I have only made new Metadata for my Free items.

I would have thought that if you select the product in the Product list and opened the Database Editor, then you should be able to amend the info in there and then save it. That's what I do if I have made a mistake.

Sorry I can't help more.

kitakoredaz

Haslor

Hello there all,

The answer is Yes, I will share my MD knowledge with anyone.
I have been working on a tutorial and and wilmap has working with a prerelease version of it.

I'm answering your PM here shortly.

Haslor

Haslor

The issue is that Metadata is not a Clean and Tidy thing AND there are a lot of Variables and it is part of the reason why you can Script some of it but never all of it. it isn't do X and everything will be right with the world. it could be X or Y or maybe in some cases Z, (but only if it is Tuesday and the wind is over 100 kph). But there are some Got-ya's that are substantial.

This has lead to a bunch of dis-information and BAD information about Metadata and the like. "Oy! Vay!" has there been a lot of that, so much so that I've been writing, rewriting and refining a tutorial for it for over a year. And Like my post on such things it is LONG, because there is a LOT to cover and it is easy to step out not he wrong path.

But once you understand it, making Metadata for a anything is Quick. I did several of Wilmap's products in under ten minutes each.

DAZ 3D has also improved a few tools inside DAZ Studio since version 4, to help with one of the Most Time Consuming parts, Scene ID's.

I am going to take some time to craft a response to the questions in this whole Thread. it will take me a day to do so. (Real work get in the way of things and I still have to pay the bills).

As for my Tutorial, on Metadata Creation for DAZ Studio 4.6, I will be releasing it soon.

Haslor.

linvanchene

Terminology:

"metadata": On the DAZ forum a term for any kind of tags, categories, scene ids and any other data that is used to organize DAZ content in DS.

"User Data": Is a collection of all CHANGES a DS user made to the metadata. By default those changes are saved to a database file. Those changes can be exported and saved to a single file located in /runtime/support.

"User Product Data": Is a term I use to describe the .dsa and .dsx files the user created by exporting the metadata of a specific product from the Product Library Content Database Editor to /runtime/support.

- - -

@ original question:

As far as I believe this is how it works:

All CHANGES you make to DAZ provided meta data is saved as User Data in a separate location in the database..

This means it does not matter if you do the changes in Product Library Content DB Editor and then export the .dsa and .dsx files OR if you just make the changes in the Content Libraries, Categories or DAZ Studio Libraries.

Any changes you make will be treated as User Data no matter where you made those changes.,

It seems as far as the database is concerned just exporting any information does not change or remove the entry in the "UserData" database location.

BUT

The moment you import your own User Product Data .dsa and .dsx files from runtime/support all this meta data will not anymore be treated as "User Data" but as originally created metadata.

Basically DAZ Studio cannot tell the difference if the imported .dsa and .dsx files were created by DAZ or the user.

There is also a visual help available.

All user created categories will have a "U" like symbol in front of them.

BUT you can observe that "U" symbol to dissappear as soon as you import your own .dsa and .dsx files.

I guess the "U" stands for "User".

My guess is that as soon as import your own .dsa and .dsx files the information in those files will not anymore have a duplicate in the "UserData" database.

Example:
You may be able to observe this if you reinstall DAZ content on a new workstation:
- If you export your "User Data" by going to Metadata Maintenance "Export Userdata" and import that single file again all your categories you created will have the "U" again.

DS seems to be able to regognize the "UserData" file as such and update that information agian in the coresponding place in the UserDatabase.

- If you go to the Product Library and open up the Content DB Editor to export your custom metadata to individual .dsa and .dsx "UserProductData" files and import those again you will notice that those categories you created will NOT anymore have an U in front of them.

DS does currently not have a way to detect if the .dsa and .dsx files containing the complete product data for an individual product were created by a user or by DAZ staff. Therefore all this information is treated as "official" metadata.

- - -

@ Additional information - Why we need more advanced metadata management options:

As long as the users only use the option "export User Data" in the content DB Editor the system works for those users who only want to make some small changes like creating their own cateogires.

Nevertheless for all users who want to make advancenced changes and are familiar with the Content DB Editor just saving out the "User Data" will not provide a backup of all the work you have done.

As it has been explained numerous times the "User Data" is not enough because it only includes the CHANGES and not the COMPLETE STATE" of the metadata.

Example:
A Daz Product is still in the category "Shoes" that is obsolete since summer 2012.
The user fixes this by putting the product in the cateogry "Footwear".
DAZ provides an update to the product but does not update the metadata. The product now is again in the category "Shoes" and "Footwear". This is NOT what the user wanted. The user does not want that category "Shoes" to appear ever again because all additoinal categories make the whole database confusing with unneccessary and outdated duplicate entries..

In addition the current "UserData" is useless for any advanced work because it is a single file for all changes.

What user need to install and manage their changes are individual .dsa and .dsx files for each product they customized!!!.

Therefore if users want a backup of all the hard work they did they need to export the .dsa and .dsx files from the Product Library / Content DB Editor to /runtime/support and then make a backup of all those .dsa and .dsx files in another location (!).

If you do not create a backup all your work will be messed up again the next time you install an updated DAZ version of the product.
As of now there still does not seem an option in the DIM to prevent that DAZ metadata is overwritting the "User Product Data" .dsa and .dsx files located in /runtime/support.

Because of that the users need a backup of their own .dsa and .dsx files that they can install again AFTER the DIM just has overwritten their changes in the database.

- - -

Solutions:

STEP 1: Accept that for some users the ability to edit and create metadata is the reason they shop in the DAZ store and use DAZ Studio

If users spend thousands of dollars on 3D content they want to be able to find that content again and organise that content in a way that makes sense to them. The current solutions are not enough to fully support the needs of those users.

STEP 2: Start working on solutions that acknowledge that the .dsa and .dsx files the users save should not be overwritten by the .dsa and .dsx files DIM updates install.

This means:

- There need to be two official save locations for metadata..

Suggestion:
/Runtime/Support for metadata .dsa and .dsx files provided by DAZ
/Runtime/UserProductData for .dsa and .dsx files created by the User

- If the DIM installs .dsa and .dsx files they are placed in /runtime/support as before
- IF the user exports .dsa and .dsx files they are placed in /runtime/UserProductData

- Iin the DIM a checkbox can be added for each product to choose if the DIM should use the DAZ .dsa and .dsx files or the "UserProductData" .dsa and .dsx files when updating

- In the Content DB Maintenance the option "export User Product Data" is added

The option "Export User Product Data" writes the .dsa and .dsx files of every product the user chooses to /runtime/UserProductData in one batch operation.

- - -

The underlying issue always seemed that the valentina CMS just does not provide the options needed to make this happen.

Now that valentina is gone I hope to see those changes implemented with PostgreSQL

Takeo.Kensei

If I understood the question, you make some metadatas and want to be sure any DAZ Product update doesn't overwrite your change

Simply put : I don't really know if user datas would be overwritten but that is something easy to check.

But if you want to be safe, you can simply export user datas before updating any product then re-import them after the update

For the other part of the debate, I think DS support should be able to answer that question as they should know what happens when products are updated
I didn't experiment to be sure, but I think that by any update, the updated products get new dsx files in My Library\Runtime\Support which will be processed to write in the content database

There is a good chance that if you customized the compatibility, it will be overwritten by the new metadatas unless DAZ provided a mechanism to prevent that

kitakoredaz

so after all, design was  problem .  

Takeo.Kensei

Just did a quick test and it works

1 - In contentDB Editor change compatibility of a product. Change also the product category and some text
2 - Export User Data with Content DB Maintenance
3 - Import back only the product metadata => Check => All my changes are gone
4 - Import ONLY User meta data with Content DB Maintenance with "User Data Overrides Product Data" checked (At the bottom) => Check the product metadata OK my changes are all back there

I don't know why it doesn't work for you but for me it worked
DS version tested : 4.6.3.29
I also use the PostgreSQL Database

[Edit] Second quick test : I reimported User Meta Data AND Product metadata at the same time with "User Data Override Product Data" checked. It still works even when reimporting both data in one go

Haslor

kitakoredaz said:

show simple case..

it is my user_data.dsx which I exported before.
now you can see, there is Amaari clothings. and each cr2. have correct compatiblity and, comaptiblity base too.

then,, I delete the prodcut.dsx,(which I modified and exported)
and install the product.dsx (daz original empty un-useful dsx packaged in zip)
and import the user_data.dsx with overwrite product meta-data option.

but it never return these compatiblity.
I tried same thing again and again,, but untill I re-modify and add compatiblity,in product,
it never work..

even though there are recoreded in user_data.dsx
about somecase, it can never be imported in database correctly.

kitakoredaz

Are you trying to create or edit your User_Data.dsx files outside of DAZ Studio's Content DB Editor?
Because what I'm seeing is a disconnect between the Asset Scene ID and the files.
I am betting if you go back to the Product Tab and look at the Scene IDs area it will be blank.
Also do you have files under Support Files?

The Scene ID is the linch pin of DAZ Metadata and the CMS Compatibility Base.
Also if you Change the GUID, you have a completely NEW Product.
If you use a GUID from another product you will break more than your data, you may break other products as well.

I am taking when you say product.dsx you means the Metadata XML file that is created by DAZ 3D and you User_Data.dsx is the datafile you have created.

There is more to Metadata than building the file, there are things that have to happen inside DAZ Studio, that have to be done inside DAZ Studio.

Haslor

kitakoredaz

metadata is only useful when daz offered correctly. 

Takeo.Kensei

There shouldn't be any problem when you create your own metadatas when no metadata exist. Even if you use an ID from another product, in worst case, you will have a duplicate ID. which can be solved with Richard Haseltine' script.

For products that already have metadatas, as I also don't see how DS could differenciate between your hand made dsx and an original dsx, the only way is to always make modifications inside DS.

If it doesn't work for you, it may mean that some datas are not well tagged in the database. The only way to know is to have a look inside the database to see if everything is alright there

There should be some way to export the database tables as flat readable text files, for analyse, eventually correct them, then import them back.

kitakoredaz

Haslor

Before the DIM if a Product didn't have Metadata, a Product.dsx didn't exist.
With the advent of the DIM, The installer has a the Product.dsx which only has the files associated with that product, but does not have the remainder of the MD create. It also doesn't have a Scene ID. One of the things this did was assign a GUID to each product, whether is had MD or not.

Now if you are replacing the DAZ 3D DSX file with your own, I am betting they have different GUID numbers and that can mess things up.

SO... do me a favor and compare the GUID's of the new record to your old version, I am betting their different.
Do yourself a favor and use the GUID which the DIM provided.
Once of the things you could do is Copy the GUID from the DAZ Product.dsx and paste it into your .dsx file, that might help you.
Also it could be that when the DAZ file in loaded the Geometry Asset's Scene IDs get removed from the Compatibility base.

One other point, are you just changing the files and then reimporting
OR are you
Deleting the installed metadata and Product and then reimporting.
The second way is the correct way, the first way will leave orphans, which will again cause you problems.

--------===== IMPORTANT SAFETY NOTE ======--------

Anyone creating or Correcting Metadata for a DAZ 3D Product:

DO NOT for ANY REASON CHANGE the GUID. Use the GUID supplied by DAZ 3D.

Here is why:
If you change the GUID, DAZ Studio and the CMS will see the new GUID as a New and Different Product.

One of the things you couldn't do before the DIM, or couldn't do very well, was share Scene files.
When you loaded a Poser file into Studio, DAZ Studio crated a GUID for the Poser content, but my GUID and Your GUID weren't the same.
This means my DAZ Studio version of V4 was different than yours; therefore when you loaded my Scene file all you would get is bounding boxes. This is also the reason why if you did a complete reload of your system, you could some times not open older scene files. one way I found around this is always backing up my Content\Data folder, so it would have the correct version of V4 that a Scene would call.

After the DIM and each of the DAZ 3D products is assigned it's own GUID by DAZ 3D.
Now My V4 and your V4 have the same GUID, therefore they are the same product.
This is what makes the Ready To Render scenes work, but you need to have all the product that are mentioned in the scene loaded.

This means that several people could work on a large scene, and when they are merged together, any of the machines could load it.

kitakoredaz

Haslor

The GUID is a Global Unique IDentifier.
The highlighted string in the upper right hand corner of the picture below.
I am betting that the GUID for your Metadata Created before the DIM installer and the one for the Post one are Different.

Also notice the list of files in the Support Files, that is all of the files associated with this Product.

Then notice there is a Scene ID at the bottom for the Geometry Asset.

So here is what I would do and have done several times (Several Hundred Times As I have corrected and created MD for DAZ 3D products)

Uninstall the broken Product.
Install Product into a Separate Content Library without any other product. (Important Step)
Make sure the Separate Library is at the Top of the Content directory (Important Step) (See my tutorial on the Content Directory Manager)
Load all this Products Geometry Assets into a New scene (to make sure there is NOTHING but this Product's assets in the scene)
Create New Scene IDs with the "Set Default Compatibility Base"
Use the Store Name for the Product Name, (Even if there is only one Geometry Asset)
Recreate you Metadata without changing the DAZ 3D GUID
Add the Support Files by importing the Whole Content Directory
Save the Metadata then copy the Support Directory files to a Temp holding folder.
Deinstall the content from the Separate folder
Reinstall it to you Live content Library
Copy the new MD XML file into the Live Content Library's Support folder over writing the DAZ supplied one.
Restart DAZ Studio, Delete the Product from the Content Library
Run the Content DB Maintenance tool
select the Re-Import Metadata
Deselect everything and only import the Product you created MD for.
Your Product should appear in the Product's area, with your selections.
Once you have this, use the Content DB Maintenance to Write out your User Data file.

Remember to Check User Data over rides Product Data on all future imports.

Loading it in to the Separate directory is KEY, because it means you don't have to Hunt and find all the assets which are contained within this Product. When you load a Asset from the Poser Formatted section of the Content Library, it will crate an Entry in the data Folder for that item. While you import the Support Directory, it will include that Data file. (This is IMPORTANT!) You may want to write out the MD file, then reenter the Content DB Editor once you have the Image file set up, so that your Support pull Includes these files as well.

Who's method are you using to create your metadata, cause some of them are just bad process?

kitakoredaz

kitakoredaz

Haslor

kitakoredaz said:

Thank you,, I think,, I can now make more stable product.dsx,, with right groval ID.

then,, now I still have some question.
when I export user_data, which datas are really exported from current DB records?

usually daz (or most of user) say, it is our modified data.

but when someone say Modified,
the database stored all modified records,, I have done in Content DB editor,
From It was resseted , and export it? (though I think it seems funny idea)

or check all asset tagged as "product" .recorded in DB at current,, ,
then only pick up additional record for each assets,,
(eg add new compatiblity with , or new category which have not recorded
in each assets of prodcut ?

Or there is another clear answer about user_data.dsx?

As far as I can tell, the UserData_1.dsx is saving any thing that was Modified, but is not yet part of a Product.

Some of the things I have found on my test system, are Scene that I have saved.
Material Change that are not part of a Product.
There is nothing contained within this Date that is part of a Product, (of either my making or DAZ 3D's)
But I think it is best summed up by the following two lines:

Conclusion, it appears the UserData_1.dsx are any data changes you made to your CMS, that are not associated to a product.

kitakoredaz said:

I hope to delete them too,, but when I see Data by Content DB editor,
I can not see any dsicribe about comaptiblity of Ammaar assets (geometries))

then Is their good way to modify user-data.dsx ?

Now I am thinking,, to make some big product_data for all my original assets,
(scene preset,, subset,, etc)
then reset-data base, and only import ,
all product.dsx witout user_data.

Personally I wouldn't Modify the Userdata.dsx, I would create Products.
This would be a much better plan then modifying the user data.
While there is what the assets are Compatible with, I see no link to the Geometry Assets.

For instance I am not seeing the Base Ring I made for a Prop, but I see it's material that I modified.

Creating Products, would create a .dsx file for the product and there for Remove it from the "No_Product" product.
Much safer bet, than Modifying the No Product, because As I said this does Not seem to include Geometry Assets.

another items are still not daz offered any dsx,, (about some A3 product,,etc)
so that I had already export original dsx about these products,,
and others are not sold in daz studio, (free items etc)
about these prodcut_dsx, there is no problem. (at least,, I have no plan to co-work
with same assets )

All I can say is: DAZ 3D is in the process of making Metadata for All of its products; but it takes Time, more time than converting the standalone installers into DIM installers.

There are improvements to DAZ Studio which the Dev's have made to speed thing up like the "Set Default Compatibility Base" API which will made Scene ID's for everything in the scene, selected or not. (the reason you don't want anything other than the product you are working on in a fresh scene.

The people inside DAZ 3D do not have a Magic Metadata Creation Script that they are not sharing.
They use the Exact Same Tools you and I use.
Think about how long it take you to make metadata for a Product, it takes them that long as well. (And they make mistakes too, same as you and I.)
So in-between creating and sometimes correcting the MD for new Products, they are making MD for old Products.

---- RANT Warning ----

This is why two years ago I suggested that the DAZ Studio User Base, Assist DAZ 3D with the creation of Metadata for their Products.
But to do that we would need a Singe Standard for the Typing and Categorization of Assets, which everyone follows.

Haslor

kitakoredaz