hipstercat
/
gck-map-extract-objects
1
0
Fork 0
Code Issues Pull Requests Releases 1 Wiki Activity

Compare commits

base: hipstercat:v1.0.0
Branches Tags
hipstercat:master
hipstercat:v1.0.0
...
compare: hipstercat:master
Branches Tags
hipstercat:master
hipstercat:v1.0.0

2 Commits
v1.0.0 ... master

Author SHA1 Message Date
Amazed 8a946952ad stuff, idk 2021-09-29 20:37:29 +02:00
Amazed 1477d5263e stuff 2021-05-30 00:27:11 +02:00
10 changed files with 1599 additions and 236 deletions
Show Stats Expand all files Collapse all files

119
README.md
View File

@ -1,119 +0,0 @@
Giants Map Objects Extractor
============================
This tool is used to get what objects are in a map. It outputs the objects model and coords and doesn't do anything else yet.
How it works
------------
Basically maps are just a zip file that contains a terrain file (.gti) and a meta file (.bin) which stores objects on the map.
Objects in this file are stored using the following format:
```
...
2a -> static byte which indicate the start of an object definition
model -> 4 bytes long value to indicate the model of the object
x -> 4 bytes float value
y -> 4 bytes float value
z -> 4 bytes float value
angle -> 4 bytes float value
...
```
Usage
----
```
C:\Projects\gck-map-extract-objects> python .\extract.py
usage: extract.py [-h] map_file
extract.py: error: the following arguments are required: map_file
```
Example
-------
```
extract.py "MvM - Ultimate Winter Duel.gck"
2019-05-24 11:00:00,229 - __main__ - INFO - Opening MvM - Ultimate Winter Duel.gck
2019-05-24 11:00:00,230 - __main__ - INFO - Found file default.gti
2019-05-24 11:00:00,230 - __main__ - INFO - Found file w_M_Mecc_Ultimate Winter Duel.bin
2019-05-24 11:00:00,230 - __main__ - INFO - Found BIN file w_M_Mecc_Ultimate Winter Duel.bin
2019-05-24 11:00:00,230 - __main__ - INFO - Opening w_M_Mecc_Ultimate Winter Duel.bin
2019-05-24 11:00:00,231 - __main__ - INFO - Object model=49 x=-469.0 y=-859.0 z=0.0 angle=0.0
2019-05-24 11:00:00,231 - __main__ - INFO - Object model=679 x=-7.300000190734863 y=-652.510009765625 z=83.86000061035156 angle=-35.0
2019-05-24 11:00:00,231 - __main__ - INFO - Object model=679 x=-19.989999771118164 y=-88.2699966430664 z=4.239999771118164 angle=0.0
2019-05-24 11:00:00,231 - __main__ - INFO - Object model=1319 x=-195.02000427246094 y=-850.3099975585938 z=175.27999877929688 angle=0.0
2019-05-24 11:00:00,231 - __main__ - INFO - Object model=679 x=-188.91000366210938 y=-23.510000228881836 z=4.239999771118164 angle=0.0
2019-05-24 11:00:00,231 - __main__ - INFO - Object model=679 x=193.85000610351562 y=-23.510000228881836 z=4.239999771118164 angle=0.0
2019-05-24 11:00:00,231 - __main__ - INFO - Object model=679 x=-271.79998779296875 y=-937.969970703125 z=180.5800018310547 angle=-312.0
2019-05-24 11:00:00,231 - __main__ - INFO - Object model=1285 x=-4.340000152587891 y=108.41000366210938 z=33.72999954223633 angle=0.0
2019-05-24 11:00:00,231 - __main__ - INFO - Object model=522 x=18.65999984741211 y=116.0999984741211 z=34.0099983215332 angle=0.0
2019-05-24 11:00:00,231 - __main__ - INFO - Object model=679 x=323.6099853515625 y=-876.6900024414062 z=201.82000732421875 angle=-151.6699981689453
2019-05-24 11:00:00,231 - __main__ - INFO - Object model=1319 x=344.6300048828125 y=-904.1099853515625 z=201.82000732421875 angle=0.0
2019-05-24 11:00:00,231 - __main__ - INFO - Object model=679 x=-260.9599914550781 y=132.4199981689453 z=4.239999771118164 angle=0.0
2019-05-24 11:00:00,231 - __main__ - INFO - Object model=679 x=-153.16000366210938 y=-1101.010009765625 z=83.86000061035156 angle=-322.6700134277344
2019-05-24 11:00:00,231 - __main__ - INFO - Object model=679 x=-172.30999755859375 y=273.4100036621094 z=4.239999771118164 angle=0.0
2019-05-24 11:00:00,232 - __main__ - INFO - Object model=679 x=285.4599914550781 y=197.58999633789062 z=1.2899999618530273 angle=0.0
2019-05-24 11:00:00,232 - __main__ - INFO - Object model=1051 x=398.67999267578125 y=131.8300018310547 z=4.239999771118164 angle=0.0
2019-05-24 11:00:00,232 - __main__ - INFO - Object model=679 x=152.89999389648438 y=308.9200134277344 z=5.579999923706055 angle=0.0
2019-05-24 11:00:00,232 - __main__ - INFO - Object model=679 x=3.630000114440918 y=342.75 z=4.239999771118164 angle=0.0
2019-05-24 11:00:00,232 - __main__ - INFO - Object model=1319 x=711.719970703125 y=-443.30999755859375 z=48.470001220703125 angle=0.0
2019-05-24 11:00:00,232 - __main__ - INFO - Object model=679 x=6.340000152587891 y=-1206.72998046875 z=83.86000061035156 angle=-191.0
2019-05-24 11:00:00,232 - __main__ - INFO - Object model=1050 x=741.4400024414062 y=-413.0400085449219 z=48.470001220703125 angle=-127.66999816894531
2019-05-24 11:00:00,232 - __main__ - INFO - Object model=981 x=8.890000343322754 y=-1259.68994140625 z=83.86000061035156 angle=180.0
2019-05-24 11:00:00,232 - __main__ - INFO - Object model=1317 x=-893.8900146484375 y=-381.3900146484375 z=0.0 angle=0.0
2019-05-24 11:00:00,232 - __main__ - INFO - Object model=1306 x=-267.07000732421875 y=479.6000061035156 z=4.239999771118164 angle=0.0
2019-05-24 11:00:00,232 - __main__ - INFO - Object model=1050 x=-676.7000122070312 y=379.6300048828125 z=83.86000061035156 angle=81.66999816894531
2019-05-24 11:00:00,232 - __main__ - INFO - Object model=1320 x=-646.2100219726562 y=403.239990234375 z=83.86000061035156 angle=0.0
2019-05-24 11:00:00,232 - __main__ - INFO - Object model=679 x=-399.5199890136719 y=-1388.6600341796875 z=169.30999755859375 angle=-207.3300018310547
2019-05-24 11:00:00,232 - __main__ - INFO - Object model=1317 x=994.530029296875 y=-537.030029296875 z=0.0 angle=0.0
2019-05-24 11:00:00,235 - __main__ - INFO - Object model=257 x=-497.5 y=685.260009765625 z=48.130001068115234 angle=56.0
2019-05-24 11:00:00,235 - __main__ - INFO - Object model=1308 x=-382.2799987792969 y=-1584.050048828125 z=93.37999725341797 angle=0.0
2019-05-24 11:00:00,235 - __main__ - INFO - Object model=679 x=139.1699981689453 y=-1660.030029296875 z=269.6499938964844 angle=-191.0
2019-05-24 11:00:00,235 - __main__ - INFO - Object model=1319 x=178.2899932861328 y=-1661.4200439453125 z=269.6499938964844 angle=0.0
2019-05-24 11:00:00,235 - __main__ - INFO - Object model=679 x=77.22000122070312 y=919.5399780273438 z=83.86000061035156 angle=36.66999816894531
2019-05-24 11:00:00,235 - __main__ - INFO - Object model=1319 x=-316.8699951171875 y=950.2999877929688 z=78.8499984741211 angle=0.0
2019-05-24 11:00:00,235 - __main__ - INFO - Object model=1319 x=209.67999267578125 y=1057.9200439453125 z=175.27999877929688 angle=0.0
2019-05-24 11:00:00,235 - __main__ - INFO - Object model=1317 x=1045.1500244140625 y=585.3200073242188 z=0.0 angle=0.0
2019-05-24 11:00:00,235 - __main__ - INFO - Object model=679 x=-261.4700012207031 y=1129.1199951171875 z=201.82000732421875 angle=144.6699981689453
2019-05-24 11:00:00,235 - __main__ - INFO - Object model=1319 x=-318.7300109863281 y=1154.6800537109375 z=201.82000732421875 angle=0.0
2019-05-24 11:00:00,235 - __main__ - INFO - Object model=679 x=288.1300048828125 y=1146.27001953125 z=176.6199951171875 angle=-135.6699981689453
2019-05-24 11:00:00,235 - __main__ - INFO - Object model=1317 x=869.0900268554688 y=-1661.699951171875 z=0.0 angle=0.0
2019-05-24 11:00:00,235 - __main__ - INFO - Object model=1317 x=-887.760009765625 y=929.8800048828125 z=0.0 angle=0.0
2019-05-24 11:00:00,235 - __main__ - INFO - Object model=1317 x=-983.75 y=-1688.030029296875 z=0.0 angle=0.0
2019-05-24 11:00:00,235 - __main__ - INFO - Object model=1319 x=-111.12000274658203 y=-2063.580078125 z=169.3800048828125 angle=0.0
2019-05-24 11:00:00,235 - __main__ - INFO - Object model=679 x=188.22000122070312 y=1288.4100341796875 z=83.86000061035156 angle=-135.6699981689453
2019-05-24 11:00:00,236 - __main__ - INFO - Object model=679 x=47.5 y=1457.300048828125 z=83.86000061035156 angle=-5.0
2019-05-24 11:00:00,236 - __main__ - INFO - Object model=981 x=46.77000045776367 y=1507.9599609375 z=83.86000061035156 angle=0.0
2019-05-24 11:00:00,236 - __main__ - INFO - Object model=679 x=468.04998779296875 y=1627.6600341796875 z=169.3800048828125 angle=-77.66999816894531
2019-05-24 11:00:00,236 - __main__ - INFO - Object model=1317 x=64.48999786376953 y=-2515.27001953125 z=0.0 angle=0.0
2019-05-24 11:00:00,236 - __main__ - INFO - Object model=1308 x=422.0299987792969 y=1821.6500244140625 z=89.43000030517578 angle=0.0
2019-05-24 11:00:00,236 - __main__ - INFO - Object model=679 x=-90.0 y=1902.81005859375 z=269.6499938964844 angle=-5.0
2019-05-24 11:00:00,236 - __main__ - INFO - Object model=1319 x=-70.79000091552734 y=1924.3399658203125 z=269.6499938964844 angle=0.0
2019-05-24 11:00:00,236 - __main__ - INFO - Object model=1317 x=1032.449951171875 y=1683.0 z=0.0 angle=0.0
2019-05-24 11:00:00,236 - __main__ - INFO - Object model=1317 x=-779.9600219726562 y=2074.550048828125 z=0.0 angle=0.0
2019-05-24 11:00:00,236 - __main__ - INFO - Object model=1317 x=328.82000732421875 y=2709.030029296875 z=0.0 angle=0.0
2019-05-24 11:00:00,236 - __main__ - INFO - Object model=830 x=61.81365966796875 y=43.321990966796875 z=33.72548294067383 angle=30.0
2019-05-24 11:00:00,236 - __main__ - INFO - Object model=830 x=-69.09912109375 y=63.499755859375 z=33.72548294067383 angle=270.0
2019-05-24 11:00:00,236 - __main__ - INFO - Object model=830 x=111.51863098144531 y=117.34283447265625 z=33.72548294067383 angle=180.0
2019-05-24 11:00:00,236 - __main__ - INFO - Object model=830 x=99.72610473632812 y=189.97406005859375 z=33.72548294067383 angle=59.0
2019-05-24 11:00:00,236 - __main__ - INFO - Object model=830 x=-57.904052734375 y=136.96914672851562 z=33.72548294067383 angle=80.0
2019-05-24 11:00:00,236 - __main__ - INFO - Object model=830 x=-30.401458740234375 y=187.08465576171875 z=33.72548294067383 angle=270.0
2019-05-24 11:00:00,236 - __main__ - INFO - Object model=830 x=-15.68878173828125 y=63.614349365234375 z=33.72548294067383 angle=180.0
2019-05-24 11:00:00,236 - __main__ - INFO - Object model=830 x=37.35343933105469 y=188.96652221679688 z=33.72548294067383 angle=270.0
2019-05-24 11:00:00,237 - __main__ - INFO - Number of objects: 62
2019-05-24 11:00:00,237 - __main__ - INFO - Model 679 was found 22 times
2019-05-24 11:00:00,237 - __main__ - INFO - Model 1317 was found 10 times
2019-05-24 11:00:00,237 - __main__ - INFO - Model 1319 was found 9 times
2019-05-24 11:00:00,237 - __main__ - INFO - Model 830 was found 8 times
2019-05-24 11:00:00,237 - __main__ - INFO - Model 1050 was found 2 times
2019-05-24 11:00:00,237 - __main__ - INFO - Model 981 was found 2 times
2019-05-24 11:00:00,237 - __main__ - INFO - Model 1308 was found 2 times
2019-05-24 11:00:00,237 - __main__ - INFO - Model 49 was found 1 times
2019-05-24 11:00:00,237 - __main__ - INFO - Model 1285 was found 1 times
2019-05-24 11:00:00,237 - __main__ - INFO - Model 522 was found 1 times
2019-05-24 11:00:00,237 - __main__ - INFO - Model 1051 was found 1 times
2019-05-24 11:00:00,237 - __main__ - INFO - Model 1306 was found 1 times
2019-05-24 11:00:00,237 - __main__ - INFO - Model 1320 was found 1 times
2019-05-24 11:00:00,237 - __main__ - INFO - Model 257 was found 1 times
```

51
anm.py
View File

@ -1,15 +1,60 @@
from lib.gbs import *
from lib.skn_anm import *
import os
def read_skn(filepath) -> AnmAnim:
def read_anm(filepath) -> AnmAnim:
with open(filepath, "rb") as fp:
init_filetrack(os.fstat(fp.fileno()).st_size)
r = AnmFile.parse(fp, AnmAnim)
return r
def read_skn(filepath) -> AnmSkin:
with open(filepath, "rb") as fp:
init_filetrack(os.fstat(fp.fileno()).st_size)
r = AnmFile.parse(fp, AnmSkin)
print("Read %s%% of file" % stat_track())
return r
def rotate_vec3_by_quaternion(vec: Vec3, quat: Quat):
u = Vec3(quat.x, quat.y, quat.z)
s = quat.w
return 2 * Vec3.dot(u, vec) * u + (s*s - Vec3.dot(u, u)) * vec + 2.0 * s * cross(u, vec)
def main():
anm = read_skn("/home/tasty/Projects/gck-map-extract-objects/anm_skn/Mc_salute.anm")
pprint(anm)
skin = read_skn("/home/tasty/Projects/gck-map-extract-objects/anm_skn/rp_l0.skn")
anm = read_anm("/home/tasty/Projects/gck-map-extract-objects/anm_skn/rp_run_scan1.anm")
#anm = read_anm("/home/tasty/Projects/gck-map-extract-objects/anm_skn/kb_roar2.anm")
#pprint(skin)
print("----------------------------------------------------------------")
print("----------------------------------------------------------------")
print("----------------------------------------------------------------")
print("----------------------------------------------------------------")
print("----------------------------------------------------------------")
# pprint(anm)
animation = AnmAnimation.create(skin, anm)
gbs = GbsData()
gbs.read("/home/tasty/Projects/gck-map-extract-objects/all_gbs/rp_l0.gbs")
animation.eval(1000, gbs.vertices, 0x4c, 0)
vbase = 0
for binder in animation.binders:
for bindings in binder.bindings:
cluster = bindings.cluster
node = bindings.node
for cluster_vertex_i in range(len(cluster.vertices)):
cluster_vertex = cluster.vertices[cluster_vertex_i]
cluster_vertex += node.transform.affine.translation
cluster_vertex = rotate_vec3_by_quaternion(cluster_vertex, node.transform.affine.quat)
gbs.vertices[vbase] = cluster_vertex
vbase += 1
print(vbase)
gbs.save_obj("/home/tasty/Projects/gck-map-extract-objects/mc_eval_0.obj")
if __name__ == "__main__":

141
id_to_gbs.py Normal file
View File

@ -0,0 +1,141 @@
import os
import json
decoded_dir = "/home/tasty/Nextcloud/docs/giants_private/binlnk (compiler for xxbin files)/all/"
def _all_gbs():
all_gbs = {}
objdata = {}
objsets = {}
files = os.listdir(decoded_dir)
for file in files:
fullpath = decoded_dir + file
print("reading %s" % fullpath)
with open(fullpath) as fp:
currblock = None
curr_objset = None
curr_obj = None
lod_found = False
audio_wav = None
audio_dist = None
for line in fp.readlines():
line = line.strip()
if not line:
continue
if line == "[objdata]":
currblock = line
continue
if line == "[objset]":
currblock = line
continue
if line == "[object]":
currblock = line
continue
if line.startswith("[") and line.endswith("]"):
currblock = None
continue
if currblock == "[objdata]":
objdata_id, gbs_name = line.split(" ")
objdata_id = int(objdata_id)
if objdata_id in objdata:
# raise Exception("%s was already in objdata ?! %s" % (objdata_id, objdata[objdata_id]))
pass
gbs_name = gbs_name.replace("\"", "")
print("OBJDATA[%s] = %s" % (objdata_id, gbs_name))
objdata[objdata_id] = gbs_name
if currblock == "[objset]":
line_attrs = line.split(" ")
if line_attrs[0] == "ID":
curr_objset = int(line_attrs[1])
lod_found = False
if line_attrs[0] == "LOD" and not lod_found:
lod_found = True
objset_objdata = int(line_attrs[1])
print("OBJSETS[%s] = %s" % (curr_objset, objset_objdata))
objsets[curr_objset] = objset_objdata
if currblock == "[object]":
line_attrs = line.split(" ")
if line_attrs[0] == "ID":
curr_obj = int(line_attrs[1])
if line_attrs[0] == "OS":
obj_objset = int(line_attrs[1])
if line_attrs[0] == "AmbientStreamLoop":
audio_wav = line_attrs[2].strip("\"")
audio_dist = float(line_attrs[4])
if line_attrs[0] == "Done":
try:
print("OBJ[%s] = OBJSETS[%s] = OBJDATA[%s] = %s" % (
curr_obj, obj_objset, objsets[obj_objset], objdata[objsets[obj_objset]]))
all_gbs[curr_obj] = {"model": objdata[objsets[obj_objset]]}
if audio_wav:
all_gbs[curr_obj]["audio"] = audio_wav
all_gbs[curr_obj]["audiodist"] = audio_dist
except KeyError:
print("ERR: could not find OBJSET %s" % obj_objset)
continue
audio_wav = None
audio_dist = None
return all_gbs
def map_txt_to_json(map_txt_path):
all_objs = []
curr_obj = None
with open(map_txt_path) as fp:
for line in fp.readlines():
line = line.strip()
if not line:
continue
line_attrs = line.split(" ")
if line_attrs[0] == "ObjectRef6":
if curr_obj:
all_objs.append(curr_obj)
curr_obj = {
"id": int(line_attrs[1]),
"x": float(line_attrs[2]),
"y": float(line_attrs[3]),
"z": float(line_attrs[4]),
"angle": float(line_attrs[5]),
"angle_2": float(line_attrs[6]),
"angle_3": float(line_attrs[7]),
"scale": 1
}
if line_attrs[0] == "ObjectRef":
if curr_obj:
all_objs.append(curr_obj)
curr_obj = {
"id": int(line_attrs[1]),
"x": float(line_attrs[2]),
"y": float(line_attrs[3]),
"z": float(line_attrs[4]),
"angle": float(line_attrs[5]),
"angle_2": 0,
"angle_3": 0,
"scale": 1
}
if line_attrs[0] == "Scale":
curr_obj["scale"] = float(line_attrs[1])
with open(map_txt_path+".json", "w") as fp:
json.dump(all_objs, fp)
return all_objs
def create_id_to_gbs_json():
g = _all_gbs()
with open("id_to_gbs.json", "w") as fp:
json.dump(g, fp)
if __name__ == '__main__':
create_id_to_gbs_json()
# m = map_txt_to_json("/home/tasty/Projects/Giants/assets/terrains/square_one_1/w_M_3Way_Tigs - Threeway - Square One.bin.txt")
# print(m)

62
lib/fileutils.py
View File

@ -1,19 +1,68 @@
import struct
file_track = []
print_debug = False
def debug_print(s):
if print_debug:
print(s)
def init_filetrack(size):
global file_track
file_track = [0] * size
def show_filetrack():
i = 0
curr = 1
curr_len = 0
while i < len(file_track):
if file_track[i] != curr:
if curr == 0:
print("%s starts at %s, ends at %s for %s (%s) bytes (%s ints)" % (curr, i-curr_len, i, curr_len, hex(curr_len), curr_len/4))
curr = file_track[i]
curr_len = 1
else:
curr_len += 1
i += 1
print(file_track)
def advance_track(offset, size):
if not file_track:
return
for i in range(size):
file_track[offset+i] = 1
def stat_track():
r = 0
total = len(file_track)
for i in file_track:
if i == 1:
r += 1
return r/total*100
def read_int(fp):
advance_track(fp.tell(), 4)
return struct.unpack("<L", fp.read(4))[0]
def read_byte(fp):
advance_track(fp.tell(), 1)
return struct.unpack("<B", fp.read(1))[0]
def read_float(fp):
advance_track(fp.tell(), 4)
return struct.unpack("<f", fp.read(4))[0]
def read_short(fp):
advance_track(fp.tell(), 2)
return struct.unpack("<H", fp.read(2))[0]
@ -22,6 +71,7 @@ def read_string(fp, size):
def read_bytes(fp, size):
advance_track(fp.tell(), size)
return bytes(struct.unpack('<' + str(size) + 'B', fp.read(size)))
@ -74,29 +124,29 @@ def decompress(compressed_bytes, original_size: int):
def read_int_and_print(n, b):
a = read_int(b)
print("%s=%s" % (n, a))
debug_print("%s=%s" % (n, a))
return a
def read_float_and_print(n, b):
a = read_float(b)
print("%s=%s" % (n, a))
debug_print("%s=%s" % (n, a))
return a
def read_byte_and_print(n, b):
a = read_byte(b)
print("%s=%s" % (n, a))
debug_print("%s=%s" % (n, a))
return a
def read_short_and_print(n, b):
a = read_short(b)
print("%s=%s" % (n, a))
debug_print("%s=%s" % (n, a))
return a
def read_string_until_none_and_print(n, b):
s = read_string_until_none(b)
print("%s=%s" % (n, s))
return s
debug_print("%s=%s" % (n, s))
return s

1023
lib/skn_anm.py
View File

File diff suppressed because it is too large Load Diff

32
map2obj.py Normal file
View File

@ -0,0 +1,32 @@
from giantslib.game.map import Map
def main():
basedir = "/home/tasty/Projects/gck-map-extract-objects/intro_island/"
map_path = basedir + "intro_island.zip"
m = Map(map_path)
m.save_heightmap(basedir + "heightmap.png")
m.save_colormap(basedir + "lightmap.png")
print(m.minheight, m.maxheight, m.stretch)
return
indexes = []
for tri in m.triangles:
index = m.vertices.index(tri)
indexes.append(index)
print(indexes)
assert(len(indexes) % 3 == 0)
assert(max(indexes) < len(m.vertices))
with open("/home/tasty/Projects/Giants/assets/terrains/test.obj", "w") as fp:
for v in m.vertices:
fp.write("v %s %s %s\n" % (v[0], v[1], v[2]))
t = 0
while t < len(indexes):
fp.write("f %s/%s %s/%s %s/%s\n" % (indexes[t]+1, indexes[t]+1, indexes[t+1]+1, indexes[t+1]+1, indexes[t+2]+1, indexes[t+2]+1))
t += 3
if __name__ == '__main__':
main()

78
obj2gbs.py
View File

@ -16,6 +16,7 @@ GBSFlagMaxLit = (1 << 31)
def check(gbs_file, materials):
return
with open("data.json") as fp:
d = json.load(fp)
@ -109,6 +110,8 @@ class OBJObject:
self.faces: List[OBJFace] = []
self.name = "root"
self.material: Union[None, OBJMaterial] = None
self.vref_start = 0
self.vref_count = 0
class OBJFace:
@ -126,14 +129,19 @@ def obj_read_materials(matlib_file) -> List[OBJMaterial]:
line = line.strip()
arr = line.split(" ")
if arr[0] == "newmtl":
if len(arr) <= 1:
curr_mat = None
continue
mat = OBJMaterial()
materials.append(mat)
mat.name = arr[1].rstrip()
curr_mat = mat
if arr[0] == "map_Ka" or arr[0] == "map_Kd":
matname_without_ext = "".join(arr[1:]).split("/")[-1]
matname_without_ext = "".join(matname_without_ext.split(".")[0:-1])
curr_mat.texture = matname_without_ext
if curr_mat:
matname_without_ext = "".join(arr[1:]).split("/")[-1]
matname_without_ext = "".join(matname_without_ext.split(".")[0:-1])
curr_mat.texture = matname_without_ext
# print("Set %s to %s" % (curr_mat.texture, curr_mat.name))
return materials
@ -205,6 +213,7 @@ class GbsData:
normals: List[Vec3] = []
objects: List[OBJObject] = []
root_obj = OBJObject()
max_objs: List[MaxObj] = []
objects.append(root_obj)
last_material = None
@ -258,22 +267,45 @@ class GbsData:
f.index_normals.append(v3_normal_index)
current_object.faces.append(f)
if arr[0] == "o":
obj_name = arr[1].rstrip()
obj_line = arr[-1].rstrip()
try:
end = obj_line.index("_#_")
except ValueError:
end = None
obj_name = obj_line[0:end]
o = OBJObject()
o.name = obj_name
o.material = last_material
if end:
meta = obj_line[end+3:].split("_")
o.vref_start = int(meta[0])
o.vref_count = int(meta[1])
objects.append(o)
if len(current_object.faces) == 0:
objects.remove(current_object)
current_object = o
if arr[0] == "usemtl":
if arr[0] == "usemtl" and len(arr) > 1:
mtl_name = arr[1].rstrip()
mtl = [mat for mat in materials if mat.name == mtl_name][0]
current_object.material = mtl
last_material = mtl
if mtl_name:
mtl = [mat for mat in materials if mat.name == mtl_name][0]
current_object.material = mtl
last_material = mtl
if arr[0] == "mtllib":
matlib_file = arr[1].rstrip()
materials = obj_read_materials("%s/%s" % (os.path.dirname(obj_file), matlib_file))
obj_mat = "%s/%s" % (os.path.dirname(obj_file), matlib_file)
print(obj_mat)
materials = obj_read_materials(obj_mat)
if arr[0] == "#" and arr[1] == "maxobj":
max_obj = MaxObj()
max_obj.vstart = int(arr[2])
max_obj.vcount = int(arr[3])
max_obj.nstart = int(arr[4])
max_obj.ncount = int(arr[5])
max_obj.noffset = int(arr[6])
max_objs.append(max_obj)
num_faces = sum([len(o.faces) for o in objects])
print("%s vertices, %s uvs, %s normals, %s objects, %s materials, %s faces" % (len(vertices), len(uvs), len(normals), len(objects), len(materials), num_faces))
@ -339,15 +371,27 @@ class GbsData:
data.put_float(v.v * -1)
# max objects
data.put_long(1) # 1 big object
data.put_long(0) # vstart
data.put_long(len_vertices) # vcount
data.put_long(0) # nstart
data.put_long(0) # ncount
data.put_long(0) # noffset ???
print("There are %s max objects" % len(max_objs))
if not max_objs:
data.put_long(1) # 1 big object
data.put_long(0) # vstart
data.put_long(len_vertices) # vcount
data.put_long(0) # nstart
data.put_long(0) # ncount
data.put_long(0) # noffset ???
else:
data.put_long(len(max_objs))
for i in range(len(max_objs)):
max_obj = max_objs[i]
data.put_long(max_obj.vstart)
data.put_long(max_obj.vcount)
data.put_long(max_obj.nstart)
data.put_long(max_obj.ncount)
data.put_long(max_obj.noffset)
# start write subobjects
data.put_long(len(objects))
print("THERE ARE %s subobjects" % len(objects))
for obj in objects:
data.put_string_size(obj.name, 32)
data.put_long(0) # max obj index
@ -360,8 +404,8 @@ class GbsData:
data.put_short(face.index_uvs[1] - 1)
data.put_short(face.index_uvs[2] - 1)
data.put_long(0) # verticeref_start
data.put_long(nverts) # verticeref_count
data.put_long(obj.vref_start) # verticeref_start
data.put_long(nverts if obj.vref_count == 0 else obj.vref_count) # verticeref_count
if options & GBSFlagUVs:
data.put_string_size(obj.material.texture, 32) # texture
data.put_string_size(obj.material.texture, 32) # bump

173
ps2_read.py Normal file
View File

@ -0,0 +1,173 @@
import argparse
import os.path
from lib.fileutils import *
from lib.packet import Packet
def ftell(fp):
return hex(fp.tell())
def read_bin(_input: str, _output: str):
with open(_input, "rb") as fp:
# read 0x30 bytes
fileversion = read_int(fp)
render_data = read_int(fp)
texturesmemory_size = read_int(fp)
unk2_size = read_int(fp)
objects_size = read_int(fp)
some_bytes = read_int(fp)
fx_size = read_int(fp)
world_size = read_int(fp)
huds_size = read_int(fp)
worldflags_size = read_int(fp)
unk = read_int(fp)
unk = read_int(fp)
print("[%s] Game is %s bytes" % (ftell(fp), render_data))
read_bytes(fp, render_data)
print("[%s] World is %s bytes..." % (ftell(fp), world_size))
read_bytes(fp, world_size)
print("[%s] Texturesmemory is %s bytes..." % (ftell(fp), texturesmemory_size))
texturesmemory = read_bytes(fp, texturesmemory_size)
print("[%s] FX %s bytes..." % (ftell(fp), fx_size))
read_bytes(fp, fx_size)
print("[%s] unk2 %s bytes..." % (ftell(fp), unk2_size))
read_bytes(fp, unk2_size)
print("[%s] worldflags %s bytes..." % (ftell(fp), worldflags_size))
read_bytes(fp, worldflags_size)
print("[%s] huds %s bytes..." % (ftell(fp), huds_size))
read_bytes(fp, huds_size)
print("[%s] objects %s bytes..." % (ftell(fp), objects_size))
objects = read_bytes(fp, objects_size)
print("[%s] textures_attributes? %s bytes..." % (ftell(fp), some_bytes))
# textures_attributes = read_bytes(fp, some_bytes)
first_int = read_int(fp)
print("[%s] First int: %s" % (ftell(fp), first_int))
second_int = read_int(fp)
print("Second int: %s" % second_int)
with open(_output, "wb") as fp:
fp.write(texturesmemory)
print("Done")
def main():
parser = argparse.ArgumentParser()
parser.add_argument("input_bin", help="PS2 bin file to read")
parser.add_argument("output_file", help="output bin for objects")
args = parser.parse_args()
read_bin(args.input_bin, args.output_file)
def read_textures(file_input):
print("Reading %s" % file_input)
curr = 0
with open(file_input, "rb") as fp:
content = fp.read()
while curr < len(content):
byte_index = content.find(b"END-OF-FIL", curr)
if byte_index != -1:
byte_index += 0xc
print("content: %s (%s)" % (content[byte_index], hex(content[byte_index])))
while content[byte_index] == 0x2a or content[byte_index] == 0x00:
byte_index += 1
print("found texture at %s: %s %s" % (hex(byte_index), hex(content[byte_index]), hex(content[byte_index+1])))
# after CLUTS
byte_index += 0x20
p = Packet(content)
p.seek(byte_index)
# psm = p.get_short()
psm = content[byte_index]
psm = psm << 8 & 0x3f
d_comp = p.get_long()
d_comp = p.get_long()
is_compressed = d_comp >> 0x12 & 0xf
compare = d_comp >> 0xe & 0xf
if is_compressed != compare:
print(hex(d_comp))
new_compare = compare - is_compressed
if new_compare <= 0:
print("ERROOOOOOOOOR")
if new_compare == 1:
decomp_bytes = 2
elif new_compare == 2:
decomp_bytes = 3
elif new_compare == 3:
decomp_bytes = 4
else:
print("ERROOOOOOOOOOR: %s" % new_compare)
print("decomp_bytes: %s" % decomp_bytes)
print("psm: %s -> %s (compressed: %s, compare: %s)" % (psm, get_pixel_format(psm), is_compressed, compare))
curr = byte_index+1
else:
curr = len(content)
def get_pixel_format(psm) -> str:
_data = {
0: "PSMCT32",
1: "PSMCT24",
2: "PSMCT16",
10: "PSMCT16S",
19: "PSMT8",
20: "PSMT4",
27: "PSMT8H",
26: "PSMT4HL",
44: "PSMT4HH",
48: "PSMZ32",
49: "PSMZ24",
50: "PSMZ16",
58: "PSMZ16S",
}
return _data.get(psm, "UNKNOWN")
def read_sizes():
print("=====================================================================================")
print("=====================================================================================")
print("=====================================================================================")
print("=====================================================================================")
print("=====================================================================================")
print("=====================================================================================")
print("=====================================================================================")
parser = argparse.ArgumentParser()
parser.add_argument("input_bin", help="PS2 bin file to read")
args = parser.parse_args()
all_num = 0
if os.path.isdir(args.input_bin):
for currdir, dirs, files in os.walk(args.input_bin):
for file in files:
if file.lower().endswith(".bin"):
read_textures(currdir+"/"+file)
else:
read_textures(args.input_bin)
print("textures:%s" % all_num)
if __name__ == '__main__':
# main()
read_sizes()

6
read_gzp.py
View File

@ -49,9 +49,9 @@ def extract_gzp(gzp_file, extensions, output_dir):
logger.info("Checksum OK")
meta_info_offset = read_int(gzp_fp)
gzp_fp.seek(meta_info_offset)
unk = read_int(gzp_fp)
print(unk)
entries_count = read_int(gzp_fp)
if entries_count == 0:
@ -69,6 +69,9 @@ def extract_gzp(gzp_file, extensions, output_dir):
compression = read_byte(gzp_fp) # compression: 1 if compressed else 0
name_length = read_byte(gzp_fp)
name = read_bytes(gzp_fp, name_length).decode("utf8").strip('\x00')
file_without_ext = ".".join(name.split(".")[0:-1])
ext = name.split(".")[-1]
logger.info(name + " compression: " + str(compression) + ", filesize: " + sizeof_fmt(
original_size) + ", start: " + str(content_offset))
@ -91,6 +94,7 @@ def extract_gzp(gzp_file, extensions, output_dir):
logger.info("File is compressed, decompressing it")
buffer = decompress(buffer, original_size)
name = "%s.%s.%s" % (file_without_ext, os.path.basename(gzp_file), ext)
logger.info("Writing file " + name)
if not os.path.exists(output_dir):
os.mkdir(output_dir)

150
skn.py
View File

@ -1,14 +1,20 @@
from lib.skn_anm import *
from lib.gbs import *
import struct
import os
import shutil
def read_skn(filepath) -> AnmSkin:
with open(filepath, "rb") as fp:
init_filetrack(os.fstat(fp.fileno()).st_size)
r = AnmFile.parse(fp, AnmSkin)
print("Read %s%% of file" % stat_track())
return r
def main():
f = ["/home/tasty/Projects/gck-map-extract-objects/anm_skn/rp_l0.skn"]
def test():
f = ["/home/tasty/Projects/gck-map-extract-objects/anm_skn/Verm_l0.skn"]
my_vertex = Vec3(-0.927200, 0.099500, 3.108000)
mini_dist_v = Vec3(0, 0, 0)
for p in f:
@ -24,5 +30,143 @@ def main():
print("%s: dist=%s" % (mini_dist_v, mini_dist_v.distance(my_vertex)))
def print_vertices(file):
skn = read_skn(file)
pprint(skn)
for obj_name in skn.default_obj_dictionnary:
obj = skn.default_obj_dictionnary[obj_name]
for cluster in obj.clusters:
for vertex in cluster.vertices:
print("v %s %s %s" % (vertex.x, vertex.y, vertex.z))
def link_gbs_skn():
basepath = "/home/tasty/Projects/gck-map-extract-objects/test/rp_l0.apatch.gzp"
skn = read_skn(basepath+".skn")
gbs = GbsData()
gbs.read(basepath+".gbs")
vbase = 0
with open("test.obj", "w") as fp:
for obj_name, obj in skn.default_obj_dictionnary.items():
for cluster in obj.clusters:
for vertex in cluster.vertices:
base_vertex_i = vertex.index + vbase
base_vertex = gbs.vertices[base_vertex_i]
new_vertex = Vec3(base_vertex.x + vertex.x, base_vertex.y + vertex.y, base_vertex.z + vertex.z)
fp.write("v %s %s %s\n" % (new_vertex.x, new_vertex.y, new_vertex.z))
vbase += obj.total_number_vertices
def main(file):
skn = read_skn(file)
pprint(skn)
vbase = 0
for tree in skn.tree_dictionnary:
for obj_name in skn.tree_dictionnary[tree]:
obj = skn.tree_dictionnary[tree][obj_name]
for cluster in obj.clusters:
if cluster.num_vertices <= 0:
print("NO VERTICES")
for vertex in cluster.vertices:
pass
# print("vertex %s -> %s" % (vertex.index, vertex.index + vbase))
vbase += obj.total_number_vertices
with open("test.obj", "w") as fp:
vbase = 0
for obj_name, obj in skn.default_obj_dictionnary.items():
for cluster in obj.clusters:
## 81: part of hair
## 12: left eye
## 9/13: leg or arm
## 15/17 is a boob
if cluster.handle == 15:
# fp.write("name %s %s %s %s %s %s %s\n" % (cluster.bone_name.strip().replace(" ", ""), cluster.bounding_box[0].x, cluster.bounding_box[0].y, cluster.bounding_box[0].z, cluster.bounding_box[1].x, cluster.bounding_box[1].y, cluster.bounding_box[1].z))
for vertex in cluster.vertices:
# fp.write("index %s\n" % (vertex.index + vbase))
fp.write("%s %s %s 0 0 0\n" % (vertex.x, vertex.y, vertex.z))
vbase += obj.total_number_vertices
def replace_vertex(x1, y1, z1, x2, y2, z2, skn_file):
final = skn_file+".repl"
if not os.path.exists(final):
shutil.copy(skn_file, final)
with open(final, "rb") as fp:
fp.seek(0)
bytesarr = bytearray(fp.read())
bfind = x1+y1+z1
brepl = x2+y2+z2
bytesarr_new = bytesarr.replace(bfind, brepl)
assert(bytesarr_new != bytesarr)
with open(final, "ab+") as fp:
fp.seek(0)
fp.write(bytesarr_new)
def replace_vertices(file_input, skn_file):
with open(skn_file, "rb") as fp:
orig_bytes = bytearray(fp.read())
with open(file_input, "r") as fp:
lines = fp.readlines()
to_search = []
for line in lines:
line = line.strip()
if not line or line.startswith("#"):
continue
a = line.split(" ")
print(line)
assert(len(a) == 6)
x1 = struct.pack("<f", float(a[0]))
y1 = struct.pack("<f", float(a[1]))
z1 = struct.pack("<f", float(a[2]))
if (x1, y1, z1) in to_search:
continue
else:
to_search.append((x1, y1, z1,))
x2 = struct.pack("<f", float(a[3]))
y2 = struct.pack("<f", float(a[4]))
z2 = struct.pack("<f", float(a[5]))
bfind = x1 + y1 + z1
brepl = x2 + y2 + z2
new_bytes = orig_bytes.replace(bfind, brepl)
assert(new_bytes != orig_bytes)
orig_bytes = new_bytes
final = skn_file+".repl"
if not os.path.exists(final):
shutil.copy(skn_file, final)
with open(final, "wb") as fp:
print("writing final file %s..." % final)
fp.write(orig_bytes)
def combine_files(input, newfile):
with open(input, "r") as in_fp:
in_lines = in_fp.readlines()
with open(newfile, "r") as out_fp:
out_lines = out_fp.readlines()
assert(len(in_lines) == len(out_lines))
for i in range(len(in_lines)):
out_lines[i] = out_lines[i].replace("\n", "") + " " + in_lines[i]
with open(newfile+".combined", "w") as o_fp:
o_fp.writelines(out_lines)
if __name__ == "__main__":
main()
file = "/home/tasty/Projects/gck-map-extract-objects/test/MC_shotgun_L0.xx_mecc_flick.gzp.skn"
s = read_skn(file)
pprint(s)