initial commit

Author: ahmedosman

LICENSE

@@ -0,0 +1,61 @@
+License
+
+Software Copyright License for non-commercial scientific research purposes
+Please read carefully the following terms and conditions and any accompanying documentation before you download and/or use the SMPL-X/SMPLify-X model, data and software, (the "Model & Software"), including 3D meshes, blend weights, blend shapes, textures, software, scripts, and animations. By downloading and/or using the Model & Software (including downloading, cloning, installing, and any other use of this github repository), you acknowledge that you have read these terms and conditions, understand them, and agree to be bound by them. If you do not agree with these terms and conditions, you must not download and/or use the Model & Software. Any infringement of the terms of this agreement will automatically terminate your rights under this License
+
+Ownership / Licensees
+The Software and the associated materials has been developed at the
+
+Max Planck Institute for Intelligent Systems (hereinafter "MPI").
+
+Any copyright or patent right is owned by and proprietary material of the
+
+Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V. (hereinafter “MPG”; MPI and MPG hereinafter collectively “Max-Planck”)
+
+hereinafter the “Licensor”.
+
+License Grant
+Licensor grants you (Licensee) personally a single-user, non-exclusive, non-transferable, free of charge right:
+
+To install the Model & Software on computers owned, leased or otherwise controlled by you and/or your organization;
+To use the Model & Software for the sole purpose of performing non-commercial scientific research, non-commercial education, or non-commercial artistic projects;
+Any other use, in particular any use for commercial purposes, is prohibited. This includes, without limitation, incorporation in a commercial product, use in a commercial service, or production of other artifacts for commercial purposes. The Model & Software may not be reproduced, modified and/or made available in any form to any third party without Max-Planck’s prior written permission.
+
+The Model & Software may not be used for pornographic purposes or to generate pornographic material whether commercial or not. This license also prohibits the use of the Model & Software to train methods/algorithms/neural networks/etc. for commercial use of any kind. By downloading the Model & Software, you agree not to reverse engineer it.
+
+No Distribution
+The Model & Software and the license herein granted shall not be copied, shared, distributed, re-sold, offered for re-sale, transferred or sub-licensed in whole or in part except that you may make one copy for archive purposes only.
+
+Disclaimer of Representations and Warranties
+You expressly acknowledge and agree that the Model & Software results from basic research, is provided “AS IS”, may contain errors, and that any use of the Model & Software is at your sole risk. LICENSOR MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE MODEL & SOFTWARE, NEITHER EXPRESS NOR IMPLIED, AND THE ABSENCE OF ANY LEGAL OR ACTUAL DEFECTS, WHETHER DISCOVERABLE OR NOT. Specifically, and not to limit the foregoing, licensor makes no representations or warranties (i) regarding the merchantability or fitness for a particular purpose of the Model & Software, (ii) that the use of the Model & Software will not infringe any patents, copyrights or other intellectual property rights of a third party, and (iii) that the use of the Model & Software will not cause any damage of any kind to you or a third party.
+
+Limitation of Liability
+Because this Model & Software License Agreement qualifies as a donation, according to Section 521 of the German Civil Code (Bürgerliches Gesetzbuch – BGB) Licensor as a donor is liable for intent and gross negligence only. If the Licensor fraudulently conceals a legal or material defect, they are obliged to compensate the Licensee for the resulting damage.
+Licensor shall be liable for loss of data only up to the amount of typical recovery costs which would have arisen had proper and regular data backup measures been taken. For the avoidance of doubt Licensor shall be liable in accordance with the German Product Liability Act in the event of product liability. The foregoing applies also to Licensor’s legal representatives or assistants in performance. Any further liability shall be excluded.
+Patent claims generated through the usage of the Model & Software cannot be directed towards the copyright holders.
+The Model & Software is provided in the state of development the licensor defines. If modified or extended by Licensee, the Licensor makes no claims about the fitness of the Model & Software and is not responsible for any problems such modifications cause.
+
+No Maintenance Services
+You understand and agree that Licensor is under no obligation to provide either maintenance services, update services, notices of latent defects, or corrections of defects with regard to the Model & Software. Licensor nevertheless reserves the right to update, modify, or discontinue the Model & Software at any time.
+
+Defects of the Model & Software must be notified in writing to the Licensor with a comprehensible description of the error symptoms. The notification of the defect should enable the reproduction of the error. The Licensee is encouraged to communicate any use, results, modification or publication.
+
+Publications using the Model & Software
+You acknowledge that the Model & Software is a valuable scientific resource and agree to appropriately reference the following paper in any publication making use of the Model & Software.
+
+Citation:
+
+
+@inproceedings{STAR:ECCV:2020,
+  title = {STAR: Sparse Trained Articulated Human Body Regressor},
+  author = {Ahmed A. A. Osman, Timo Bolkart, Michael J. Black},
+  booktitle = {European Conference on Computer Vision (ECCV) },
+  month = aug,
+  year = {2020},
+  month_numeric = {8}
+}
+
+Commercial licensing opportunities
+For commercial uses of the Software, please send email to ps-license@tue.mpg.de
+
+This Agreement shall be governed by the laws of the Federal Republic of Germany except for the UN Sales Convention.

README.md

@@ -0,0 +1,106 @@
+## STAR: Sparse Trained Articulated Human Body Regressor 
+
+<!-- TODO: Replace with our arxiv link -->
+<!-- [![report](https://img.shields.io/badge/arxiv-report-red)](https://arxiv.org/abs/1912.05656) -->
+
+[[Project Page](https://star.is.tue.mpg.de/)] 
+[[Paper](https://ps.is.tuebingen.mpg.de/uploads_file/attachment/attachment/618/star_paper.pdf)]
+[[Supp. Mat.](https://ps.is.tuebingen.mpg.de/uploads_file/attachment/attachment/619/star_supmat.pdf)]
+
+<p align="center">
+  <img src="./images/main_teaser.png" />
+</p>
+
+
+## Table of Contents
+  * [License](#license)
+  * [Description](#description)
+    * [Content](#content)
+    * [Dependencies](#dependencies)
+    * [SMPL Comparison](#SMPLComparison)
+    * [Profiling](#Profiling) 
+  * [Citation](#citation)
+  * [Acknowledgments](#acknowledgments)
+  * [Contact](#contact)
+
+
+## License
+
+Software Copyright License for non-commercial scientific research purposes.
+Please read carefully the following terms and conditions and any accompanying
+documentation before you download and/or use the ExPose data, model and
+software, (the "Data & Software"), including 3D meshes, images, videos,
+textures, software, scripts, and animations. By downloading and/or using the
+Data & Software (including downloading, cloning, installing, and any other use
+of the corresponding github repository), you acknowledge that you have read
+these terms and conditions, understand them, and agree to be bound by them. If
+you do not agree with these terms and conditions, you must not download and/or
+use the Data & Software. Any infringement of the terms of this agreement will
+automatically terminate your rights under this License
+
+
+## Description
+
+STAR - **S**parse **T**rained  **A**rticulated Human Body **R**egressor is a generateive 3D human body model, that is designed to be a drop in replacement to the widely used SMPL model.
+STAR trained on largest dataset of more than 10,000 human body scans, with a learned set of sparse spatially local pose corrective 
+blend shapes. In the Figure below, a single joint movement only influence a sparse set of the model vertices. The mesh vertices in 
+gray are not affected by the joint movement, in contrast to SMPL.  <br/>
+STAR is publicly avaiable with the full 300 principal component 
+shape space for research purposes from our [website][https://star.is.tue.mpg.de/]
+
+<p align="center">
+  <img src="./images/sparse_pose_correctives.png" />
+</p>
+
+
+ For more details, please see our ECCV paper
+[STAR: Sparse Trained Articulated Human Body Regressor](https://ps.is.mpg.de/uploads_file/attachment/attachment/618/star_paper.pdf).
+
+## Content
+This repository contains the model loader in the following frameworks:
+* A PyTorch. 
+* A Tensorflow 2.0.
+* A Chumpy.
+
+Code tested on Python 3.69, CUDA 10.1, CuDNN 7.6.5 and PyTorch 1.6.0, Tensorflow 2.3 , Chumpy 0.69 on Ubuntu 18.04
+
+## SMPL Comparison 
+STAR is designed to be a drop in replacement for SMPL, similar to SMPL it is parameterised with pose and shape parameters. 
+
+<p align="center">
+  <img src="./images/star_talk amazon.053.jpeg" />
+</p>
+
+### STAR Kinematic Tree
+<p align="center">
+  <img src="./images/star_kinematic_tree.png" />
+</p>
+
+
+
+
+## Citation
+
+If you find this Model & Software useful in your research we would kindly ask you to cite:
+
+```bibtex
+@inproceedings{STAR:ECCV:2020,
+  title = {STAR: Sparse Trained Articulated Human Body Regressor},
+  author = {Ahmed A. A. Osman, Timo Bolkart, Michael J. Black},
+  booktitle = {European Conference on Computer Vision (ECCV) },
+  month = aug,
+  year = {2020},
+  month_numeric = {8}
+}
+```
+
+## Acknowledgments
+We thank Naureen M. Mahmood, Talha Zaman,  Nikos Athanasiou, Muhammed Kocabas, Nikos Kolotouros and Vassilis Choutas for the discussions. 
+and Lea Muller,  Amir Ahmad and Nitin Saini for proof reading the script and Joachim Tesch for help with game engines plug-ins.
+and Mason Laundry for the voice over.
+
+## Contact
+
+For questions, please contact [star@tue.mpg.de](mailto:star@tue.mpg.de). 
+
+For commercial licensing (and all related questions for business applications), please contact [ps-licensing@tue.mpg.de](mailto:ps-licensing@tue.mpg.de).

__init__.py

@@ -0,0 +1,20 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2020 Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V. (MPG),
+# acting on behalf of its Max Planck Institute for Intelligent Systems and the
+# Max Planck Institute for Biological Cybernetics. All rights reserved.
+#
+# Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V. (MPG) is holder of all proprietary rights
+# on this computer program. You can only use this computer program if you have closed a license agreement
+# with MPG or you get the right to use the computer program from someone who is authorized to grant you that right.
+# Any use of the computer program without a valid license is prohibited and liable to prosecution.
+# Contact: ps-license@tuebingen.mpg.de
+#
+#
+# If you use this code in a research publication please consider citing the following:
+#
+# STAR: Sparse Trained  Articulated Human Body Regressor <https://arxiv.org/pdf/2008.08535.pdf>
+#
+#
+# Code Developed by:
+# Ahmed A. A. Osman 

ch/__init__.py

@@ -0,0 +1,58 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2020 Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V. (MPG),
+# acting on behalf of its Max Planck Institute for Intelligent Systems and the
+# Max Planck Institute for Biological Cybernetics. All rights reserved.
+#
+# Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V. (MPG) is holder of all proprietary rights
+# on this computer program. You can only use this computer program if you have closed a license agreement
+# with MPG or you get the right to use the computer program from someone who is authorized to grant you that right.
+# Any use of the computer program without a valid license is prohibited and liable to prosecution.
+# Contact: ps-license@tuebingen.mpg.de
+#
+#
+# If you use this code in a research publication please consider citing the following:
+#
+# STAR: Sparse Trained  Articulated Human Body Regressor <https://arxiv.org/pdf/2008.08535.pdf>
+#
+#
+# Code Developed by:
+# Ahmed A. A. Osman 
+
+import chumpy  as ch
+import numpy as np
+import os
+from .verts import verts_decorated_quat 
+from config import cfg 
+
+def load_model(gender='female',num_betas=10):
+    if gender not in ['male','female']:
+        raise RuntimeError('Invalid model gender!')
+    fname = os.path.join(cfg.path_star,gender,'model.npy')
+    model_dict  = np.load(fname,allow_pickle=True,encoding='latin1')[()]
+    trans       = ch.array(np.zeros(3))
+    posedirs    = ch.array(model_dict['posedirs'])
+    v_tempalate = ch.array(model_dict['v_template'])
+
+    J_regressor   = model_dict['J_regressor'] #Regressor of the model
+    weights       = ch.array(model_dict['weights']) #Weights
+    num_joints    = weights.shape[1]
+    kintree_table = model_dict['kintree_table']
+    f = model_dict['f']
+
+    betas = ch.array(np.zeros((model_dict['shapedirs'].shape[-1]))) #Betas
+    shapedirs = ch.array(model_dict['shapedirs']) #Shape Corrective Blend shapes
+    pose = ch.array(np.zeros((num_joints*3))) #Pose Angles
+
+    model = verts_decorated_quat(trans=trans,
+                    pose=pose,
+                    v_template=v_tempalate,
+                    J=J_regressor,
+                    weights=weights,
+                    kintree_table=kintree_table,
+                    f=f,
+                    posedirs=posedirs,
+                    betas=betas,
+                    shapedirs=shapedirs,
+                    want_Jtr=True)
+    return model

ch/serialization.py

@@ -0,0 +1,92 @@
+#!/usr/bin/env python
+# encoding: utf-8
+# Copyright (c) 2016 Max Planck Society. All rights reserved.
+# Modified by Ahmed A. A. Osman Feb 2020 
+
+import chumpy as ch 
+import numpy as np
+import cv2
+
+def axis2quat(p):
+    angle =  ch.sqrt(ch.clip(ch.sum(ch.square(p),1), 1e-16,1e16))
+    norm_p = p / angle[:,np.newaxis]
+    cos_angle = ch.cos(angle/2)
+    sin_angle = ch.sin(angle/2)
+    qx = norm_p[:,0]*sin_angle
+    qy = norm_p[:,1]*sin_angle
+    qz = norm_p[:,2]*sin_angle
+    qw =  cos_angle -1
+    return ch.concatenate([qx[:,np.newaxis],qy[:,np.newaxis],qz[:,np.newaxis],qw[:,np.newaxis]],axis=1)
+
+
+class Rodrigues(ch.Ch):
+    dterms = 'rt'
+    
+    def compute_r(self):
+        return cv2.Rodrigues(self.rt.r)[0]
+    
+    def compute_dr_wrt(self, wrt):
+        if wrt is self.rt:
+            return cv2.Rodrigues(self.rt.r)[1].T
+
+
+def global_rigid_transformation(pose, J, kintree_table):
+    results = {}
+    pose = pose.reshape((-1, 3))
+    id_to_col = {kintree_table[1, i]: i for i in range(kintree_table.shape[1])}
+    parent = {i: id_to_col[kintree_table[0, i]] for i in range(1, kintree_table.shape[1])}
+    def with_zeros(x):
+        return ch.vstack((x, ch.array([[0.0, 0.0, 0.0, 1.0]])))
+
+    results[0] = with_zeros(ch.hstack((Rodrigues(pose[0, :]), J[0, :].reshape((3, 1)))))
+
+    for i in range(1, kintree_table.shape[1]):
+        results[i] = results[parent[i]].dot(
+            with_zeros(ch.hstack(
+                (Rodrigues(pose[i, :]), ((J[i, :] - J[parent[i], :]).reshape((3, 1))))
+            )))
+
+    def pack(x):
+        return ch.hstack([np.zeros((4, 3)), x.reshape((4, 1))])
+
+    results = [results[i] for i in sorted(results.keys())]
+    results_global = results
+
+    if True:
+        results2 = [
+            results[i] - pack(results[i].dot(ch.concatenate(((J[i, :]), 0))))
+            for i in range(len(results))]
+
+        results = results2
+
+    result = ch.dstack(results)
+    return result, results_global
+
+def verts_core(pose, v, J, weights, kintree_table, want_Jtr=False):
+    A, A_global = global_rigid_transformation(pose, J, kintree_table)
+    T = A.dot(weights.T)
+
+    rest_shape_h = ch.vstack((v.T, np.ones((1, v.shape[0]))))
+
+    v = (T[:, 0, :] * rest_shape_h[0, :].reshape((1, -1)) +
+         T[:, 1, :] * rest_shape_h[1, :].reshape((1, -1)) +
+         T[:, 2, :] * rest_shape_h[2, :].reshape((1, -1)) +
+         T[:, 3, :] * rest_shape_h[3, :].reshape((1, -1))).T
+
+    v = v[:, :3]
+
+    class result_meta(object):
+        pass
+
+    if not want_Jtr:
+        Jtr = None
+    else:
+        Jtr = ch.vstack([g[:3, 3] for g in A_global])
+
+    meta = result_meta()
+    meta.Jtr = Jtr
+    meta.A = A
+    meta.A_global = A_global
+    meta.A_weighted = T
+
+    return v, meta