aargh Posted August 11, 2018 Share Posted August 11, 2018 (edited) Hi After deleting the .meta file for a mesh, a new one is generated with a different guid. This is a problem because the references contain the original guid. How to solve this problem? I assumed that the guid is deterministically generated. An example is attached. Between versions of .meta file <asset version = "2.70"> and <asset version = "2.7.2.2"> did not change the referenced mesh file, but the hash is different. Edit: What is the algorithm for generating guid? Is it derived from a hash of file? If so, how do you solve the loss of .meta file? Especially if references already exist. Edited August 11, 2018 by aargh Link to comment
morbid Posted August 13, 2018 Share Posted August 13, 2018 Hello aargh, On 8/12/2018 at 4:02 AM, aargh said: After deleting the .meta file for a mesh, a new one is generated with a different guid This is the correct behavior, we generate guids non-deterministically. Each time .meta is lost it will be regenerated with different value. To keep all nodes you need the original meta. If node lost its material\mesh or anything else you'll need to fix it manually. On 8/12/2018 at 4:02 AM, aargh said: Between versions of .meta file <asset version = "2.70"> and <asset version = "2.7.2.2"> did not change the referenced mesh file, but the hash is different. It's also a valid case because version change usually comes with some modifications, so we need to reimport everything once after migration. On 8/12/2018 at 4:02 AM, aargh said: Edit: What is the algorithm for generating guid? Is it derived from a hash of file? Well, it's random and doesn't depend on hash sum. Thanks! How to submit a good bug report --- FTP server for test scenes and user uploads: ftp://files.unigine.com user: upload password: 6xYkd6vLYWjpW6SN Link to comment
aargh Posted August 13, 2018 Author Share Posted August 13, 2018 Hi if I delete a meta file, I will always get the same guid when creating a new one. The change only occurs between SDKs. That's why I asked about the generation algorithm Link to comment
morbid Posted August 13, 2018 Share Posted August 13, 2018 Sorry, I missed one point in the previous message. If you delete .meta file when the Editor is running - the original values for asset will be restored (not generated). If you delete .meta file when the Editor is not loaded, just in file manager, Editor will treat this file without meta as a new one and completely new GUID will be generated. let's go back to your case. What exactly happened? You wrote that .meta was deleted, if it was, then the generation of a new GUID is correct behavior. If this happened after migration - it could be a bug that needs a way to be reproduced. Thanks! How to submit a good bug report --- FTP server for test scenes and user uploads: ftp://files.unigine.com user: upload password: 6xYkd6vLYWjpW6SN Link to comment
aargh Posted August 13, 2018 Author Share Posted August 13, 2018 6 minutes ago, morbid said: Sorry, I missed one point in the previous message. If you delete .meta file when the Editor is running - the original values for asset will be restored (not generated). If you delete .meta file when the Editor is not loaded, just in file manager, Editor will treat this file without meta as a new one and completely new GUID will be generated. let's go back to your case. What exactly happened? You wrote that .meta was deleted, if it was, then the generation of a new GUID is correct behavior. If this happened after migration - it could be a bug that needs a way to be reproduced. Thanks! Hi Ad 2 the editor generates the same guid for the file. This has led me to question whether the guid is deterministic.in the appendix is original meta (first) and meta after new creation by the editor (second) files are for data\asset_library\characters\UMA\meshes\UMA_male.mesh UMA_male.mesh.meta_first UMA_male.mesh.meta_second Link to comment
demostenes Posted August 13, 2018 Share Posted August 13, 2018 40 minutes ago, morbid said: let's go back to your case. What exactly happened? You wrote that .meta was deleted, if it was, then the generation of a new GUID is correct behavior. If this happened after migration - it could be a bug that needs a way to be reproduced. Thanks! 1) It seems, that GUID generation algorithm was changed recently, correct? 2) We are facing crashes and duplicate guid issues: https://developer.unigine.com/forum/topic/4988-duplicate-guids-in-log-and-crashes/ We did all node duplications via editor, so it seems like some bug. Because editor is crashing/hanging quite often during various duplicating operations https://developer.unigine.com/forum/topic/4972-editor-sometimes-hangs-when-duplicating-node/ it leads us to suspiction, that this can be reason, why duplicated guids appeared. 3) it would be very helpful to extend logging, so we cant find, which file is problematic. Such error message is not much descriptive: Thanks! Link to comment
aargh Posted August 13, 2018 Author Share Posted August 13, 2018 And back to my case... After several crashes of the editor when copying nodes, I found this in the log 14:13:49 FileSystemAssets::load_runtime(): guid "7c1ce7913f7ea73c09945bb57383cd4ef96c8310" is already in use 14:13:49 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:49 FileSystemAssets::load_runtime(): guid "6713e5b9c2afcf3370262a5f6dbd4d1cb7599720" is already in use 14:13:50 FileSystemAssets::load_runtime(): guid "9ed63fe1805b90815509d71dd6040a5ce3d48b30" is already in use 14:13:50 FileSystemAssets::load_runtime(): guid "c05b8da7ca91d078f5fdae08f69fc8e6a827b2c0" is already in use 14:13:50 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:50 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:50 FileSystemAssets::load_runtime(): guid "9b7330ea1fcb4b362a8f12741a92b18341ab4f51" is already in use 14:13:50 FileSystemAssets::load_runtime(): unknown "4cb9dd49d33dcce01c002c56ed790273b7d838c2" asset guid 14:13:50 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:50 FileSystemAssets::load_runtime(): guid "31f78f7e9324be5428d312d272cf7b2dd709c6b1" is already in use 14:13:50 FileSystemAssets::load_runtime(): guid "d2850c6ce3bf7c897c905997cf4e7e6bcf42bdc1" is already in use 14:13:50 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:50 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:50 FileSystemAssets::load_runtime(): unknown "4cb9dd49d33dcce01c002c56ed790273b7d838c2" asset guid 14:13:50 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:50 FileSystemAssets::load_runtime(): guid "2fb6ac833535108ff3516d217529086757da2603" is already in use 14:13:50 FileSystemAssets::load_runtime(): guid "57cd839eef6f88dda51983672c9d0793c9daa333" is already in use 14:13:50 FileSystemAssets::load_runtime(): guid "65dcf2e0b3a458e574d7a3ed96c1c43c2ea45943" is already in use 14:13:50 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:50 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:50 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:50 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:50 FileSystemAssets::load_runtime(): guid "bca209fee6f3fee3653381579810cbec6d3aaa93" is already in use 14:13:50 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:50 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:50 FileSystemAssets::load_runtime(): guid "e861fd0e434b1c2be83cca2070d999ef12f6da34" is already in use 14:13:50 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:50 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:50 FileSystemAssets::load_runtime(): guid "902d8ab6f9914a6cdf18950fea13bb63d2c7e984" is already in use 14:13:50 FileSystemAssets::load_runtime(): guid "c7a2500d1fea65dd337407aa6de620b6414cf8a4" is already in use 14:13:50 FileSystemAssets::load_runtime(): guid "ffdf790627c0df05c4b04d91f34eb64b2bdd29a4" is already in use 14:13:50 FileSystemAssets::load_runtime(): guid "33204f6bd16d43c013f6bd4356c827393e19a205" is already in use 14:13:50 FileSystemAssets::load_runtime(): guid "8be192237710bae03bd98405eb27e609de8cc7f5" is already in use 14:13:50 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:50 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:50 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:50 FileSystemAssets::load_runtime(): guid "6a695da42c887a43f9b37cd8f22e3740115cc9c6" is already in use 14:13:50 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:50 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:50 FileSystemAssets::load_runtime(): guid "7fecdf313054e3ea88567dd1d1d605f8c618d817" is already in use 14:13:50 FileSystemAssets::load_runtime(): guid "dbaf863f2666844318942abe55fa7dd4f36c74c7" is already in use 14:13:50 FileSystemAssets::load_runtime(): guid "f037d9580e68fef900bf5c1d77ad808a52428bf7" is already in use 14:13:51 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:51 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:51 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:51 FileSystemAssets::load_runtime(): guid "ecc9287e6eac45e88984467088d234a0b4f68368" is already in use 14:13:51 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:51 FileSystemAssets::load_runtime(): guid "03d37f3270451c15e2d07ed9b37a4ec1030acda8" is already in use 14:13:51 FileSystemAssets::load_runtime(): guid "78ba8f215d98a91029bfab3f6fc464fc2f52d3e8" is already in use 14:13:51 FileSystemAssets::load_runtime(): guid "af8dc01daa4603bbeb8236b4d30decc6b3d22f19" is already in use 14:13:51 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:51 FileSystemAssets::load_runtime(): guid "6d82a7580a021d86848325f7546db8c8227217d9" is already in use 14:13:51 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:51 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:51 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:51 FileSystemAssets::load_runtime(): guid "c2f532af108894b16139d051a42bddac98fe136b" is already in use 14:13:51 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:51 FileSystemAssets::load_runtime(): guid "16255eb07203d4b19d5d20c6e4e4227ee77e1bdb" is already in use 14:13:51 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:51 FileSystemAssets::load_runtime(): guid "6163269213cbebea750079b089612ade934d4dbc" is already in use 14:13:51 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:51 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:51 FileSystemAssets::load_runtime(): guid "ffc02c10477ae6a3022b28e1c23a5346356b0d0d" is already in use 14:13:51 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:51 FileSystemAssets::load_runtime(): guid "15036ca7e1452c4bf6a26ac91004cf635070eb6d" is already in use 14:13:51 FileSystemAssets::load_runtime(): guid "fdc2617595584a52a05fa8b44ecdb22e070a0e8d" is already in use 14:13:51 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:51 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:51 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:51 FileSystemAssets::load_runtime(): guid "f0d451fd380480bceaacefca29c5c5287a67cd5e" is already in use 14:13:51 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:51 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:51 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:52 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid 14:13:52 FileSystemAssets::load_runtime(): guid "ddbc3d08139ea7c36354776d305387e93ebd31af" is already in use 14:13:52 FileSystemAssets::load_runtime(): unknown "f920836fab836cd224b426e9e1eb3eb959646aef" asset guid I started looking for the cause and discovered that some files are not up to date on the latest version. I reported this elsewhere.I made a change to upgrade the script to take place for all old files.Based on my assumption of guid stability, I deleted the meta files and let them re-create them. Which was a mistake.- restore the meta files from the backup- you need to find out why the file editor creates errors in files (duplicates or invalid xml)- Upgrade script only checks the first found file for the version, others skip- write a precise description of the metafile mechanics in the documentation. This would help a lot Link to comment
aargh Posted August 13, 2018 Author Share Posted August 13, 2018 (edited) Small addition I found this error on log file 14:15:21 "AssetManager::init_core_assets(): FileSystem error. GUID = e038d46e26ad35b1e7775c22e5377971790cb075, path = core/textures/sensor/lut_green.dds" 14:15:21 "AssetManager::init_core_assets(): FileSystem error. GUID = 2a3d6d41edb8ac91336265af489c891e9d772658, path = core/textures/sensor/lut_heat.dds" 14:15:21 "AssetManager::init_core_assets(): FileSystem error. GUID = 434ab133edf4b06c916e3d36bd646d04746da384, path = core/textures/sensor/lut_red.dds" 14:15:21 "AssetManager::init_core_assets(): FileSystem error. GUID = e237a1333473ddcc4729b8cfa3bee53e75991b77, path = core/textures/sensor/lut_white.dds" 14:15:21 "AssetManager::init_core_assets(): FileSystem error. GUID = a6d3c23f9fb69fffd5fe6aaa55c6a64559ba1f06, path = core/materials/default/sensor/post_sensor.basemat" 14:15:21 "AssetManager::init_core_assets(): FileSystem error. GUID = 19a181cc831bedbdefe97d193d1aa76fbac66260, path = core/materials/default/sensor/post_sensor_green.mat" 14:15:21 "AssetManager::init_core_assets(): FileSystem error. GUID = 153e6c29c36ec1a0be3922f12928e8c75d53e58b, path = core/materials/default/sensor/post_sensor_heat.mat" 14:15:21 "AssetManager::init_core_assets(): FileSystem error. GUID = 668515a810feac4b6532b0d6f2cf53947b33ca32, path = core/materials/default/sensor/post_sensor_red.mat" 14:15:21 "AssetManager::init_core_assets(): FileSystem error. GUID = 6c9028091619e0c167f3e8e32b7d067784cbfcb0, path = core/materials/default/sensor/post_sensor_white.mat" This is situation on version before my changes Please extend logging, this is cryptic output for error checking on big project 14:30:57 ----Properties---- 14:30:57 Xml::getBoolData(): unknown bool value "" in "parameter" node 14:30:57 Xml::getFloatArrayData(): can't read 0 array element in "parameter" node 14:30:57 Property::load_parameter_item(): can't read "color" parameter in "showcase" property 14:30:57 Xml::getBoolData(): unknown bool value "" in "parameter" node 14:30:58 Engine properties loaded: 2 / 2 (0ms) 14:30:58 User properties loaded: 294 / 294 (107ms) Edited August 13, 2018 by aargh Link to comment
morbid Posted August 13, 2018 Share Posted August 13, 2018 2 hours ago, aargh said: Ad 2 the editor generates the same guid for the file. This has led me to question whether the guid is deterministic. My bad, I forgot to mention that it also could be restored from guids.db file. If the required string is absent - you'll get a new GUID. 1 hour ago, demostenes said: 1) It seems, that GUID generation algorithm was changed recently, correct? Nope, it remained the same. 1 hour ago, aargh said: I started looking for the cause and discovered that some files are not up to date on the latest version. I reported this elsewhere. Okay, please check the corresponding topic. 1 hour ago, aargh said: you need to find out why the file editor creates errors in files (duplicates or invalid xml) Could you explain how exactly you're copying the assets? Copy-paste in Editor? Or in file manager? I've started your project migration from 2.7.1 to 2.7.2, we'll try to give you feedback tomorrow. Thank you! How to submit a good bug report --- FTP server for test scenes and user uploads: ftp://files.unigine.com user: upload password: 6xYkd6vLYWjpW6SN Link to comment
demostenes Posted August 13, 2018 Share Posted August 13, 2018 6 minutes ago, morbid said: Could you explain how exactly you're copying the assets? Copy-paste in Editor? Or in file manager? Only via editor. Copy paste/duplicate function. Never ever outside editor. Link to comment
morbid Posted August 14, 2018 Share Posted August 14, 2018 Hi, we've reproduced random crash with world loading you reportedearlier. The issue with lost mesh that @aargh described in the first place didn't show up. I've checked your nodes in asset_library folder. I'd like to note that migration from 2.7.1 went smoothly. We still investigating "unknown GUID" problem. Thanks! How to submit a good bug report --- FTP server for test scenes and user uploads: ftp://files.unigine.com user: upload password: 6xYkd6vLYWjpW6SN Link to comment
demostenes Posted August 14, 2018 Share Posted August 14, 2018 Thanks, thats good news. I ve taken revision of the project 463, deleted guids.db and runtime directory and run editor. I have this guid error in the log: FileSystem::mount_file(): can't mount "asset_library/rocks/rock_for_landscapes/mesh/Pile_Ramp_Small.mesh" file, because of GUID collision I ve fulltext searched this guid through project (.meta, .node, .world,.mat) but it is only in Pile_Ramp_Small.mesh.meta How is this possible? Guid DB is fresh and meta is only 1x, which is OK. Thanks. Link to comment
Recommended Posts