Dynamic Omnidirectional Texture Synthesis for Photorealistic Virtual Content Creation

Chih Fan Chen; Evan Suma Rosenberg

doi:10.1109/ISMAR-Adjunct.2018.00040

Dynamic Omnidirectional Texture Synthesis for Photorealistic Virtual Content Creation

Chih Fan Chen, Evan Suma Rosenberg

Computer Science and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

3 Scopus citations

Abstract

We present a dynamic omnidirectional texture synthesis (DOTS) approach for generating real-time virtual reality content captured using a consumer-grade RGB-D camera. Compared to a single fixed-viewpoint color map, view-dependent texture mapping (VDTM) techniques can reproduce finer detail and replicate dynamic lighting effects that become especially noticeable with head tracking in virtual reality. However, VDTM is very sensitive to errors such as missing data or inaccurate camera pose estimation, both of which are commonplace for objects captured using consumer-grade RGB-D cameras. To overcome these limitations, our proposed optimization can synthesize a high resolution view-dependent texture map for any virtual camera location. Synthetic textures are generated by uniformly sampling a spherical virtual camera set surrounding the virtual object, thereby enabling efficient real-time rendering for all potential viewing directions.

Original language	English (US)
Title of host publication	Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	85-90
Number of pages	6
ISBN (Electronic)	9781538675922
DOIs	https://doi.org/10.1109/ISMAR-Adjunct.2018.00040
State	Published - Jul 2 2018
Event	17th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018 - Munich, Germany Duration: Oct 16 2018 → Oct 20 2018

Publication series

Name	Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018

Conference

Conference	17th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018
Country/Territory	Germany
City	Munich
Period	10/16/18 → 10/20/18

Bibliographical note

Funding Information:
This work is sponsored by the U.S. Army Research Laboratory (ARL) under contract number W911NF-14-D-0005. Statements and opinions expressed and content included do not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred.

Keywords

view-dependent texture mapping
virtual content creation
virtual reality

Access

10.1109/ISMAR-Adjunct.2018.00040

OpenUrl availability

Full text

Cite this

Chen, C. F., & Rosenberg, E. S. (2018). Dynamic Omnidirectional Texture Synthesis for Photorealistic Virtual Content Creation. In Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018 (pp. 85-90). Article 8699299 (Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISMAR-Adjunct.2018.00040

Dynamic Omnidirectional Texture Synthesis for Photorealistic Virtual Content Creation. / Chen, Chih Fan; Rosenberg, Evan Suma.
Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018. Institute of Electrical and Electronics Engineers Inc., 2018. p. 85-90 8699299 (Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Chen, CF & Rosenberg, ES 2018, Dynamic Omnidirectional Texture Synthesis for Photorealistic Virtual Content Creation. in Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018., 8699299, Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018, Institute of Electrical and Electronics Engineers Inc., pp. 85-90, 17th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018, Munich, Germany, 10/16/18. https://doi.org/10.1109/ISMAR-Adjunct.2018.00040

Chen CF, Rosenberg ES. Dynamic Omnidirectional Texture Synthesis for Photorealistic Virtual Content Creation. In Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 85-90. 8699299. (Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018). doi: 10.1109/ISMAR-Adjunct.2018.00040

Chen, Chih Fan ; Rosenberg, Evan Suma. / Dynamic Omnidirectional Texture Synthesis for Photorealistic Virtual Content Creation. Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 85-90 (Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018).

@inproceedings{005eb62a21ca440da0f4c7b3937f7d66,

title = "Dynamic Omnidirectional Texture Synthesis for Photorealistic Virtual Content Creation",

abstract = "We present a dynamic omnidirectional texture synthesis (DOTS) approach for generating real-time virtual reality content captured using a consumer-grade RGB-D camera. Compared to a single fixed-viewpoint color map, view-dependent texture mapping (VDTM) techniques can reproduce finer detail and replicate dynamic lighting effects that become especially noticeable with head tracking in virtual reality. However, VDTM is very sensitive to errors such as missing data or inaccurate camera pose estimation, both of which are commonplace for objects captured using consumer-grade RGB-D cameras. To overcome these limitations, our proposed optimization can synthesize a high resolution view-dependent texture map for any virtual camera location. Synthetic textures are generated by uniformly sampling a spherical virtual camera set surrounding the virtual object, thereby enabling efficient real-time rendering for all potential viewing directions.",

keywords = "view-dependent texture mapping, virtual content creation, virtual reality",

author = "Chen, {Chih Fan} and Rosenberg, {Evan Suma}",

note = "Funding Information: This work is sponsored by the U.S. Army Research Laboratory (ARL) under contract number W911NF-14-D-0005. Statements and opinions expressed and content included do not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred.; 17th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018 ; Conference date: 16-10-2018 Through 20-10-2018",

year = "2018",

month = jul,

day = "2",

doi = "10.1109/ISMAR-Adjunct.2018.00040",

language = "English (US)",

series = "Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "85--90",

booktitle = "Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018",

}

TY - GEN

T1 - Dynamic Omnidirectional Texture Synthesis for Photorealistic Virtual Content Creation

AU - Chen, Chih Fan

AU - Rosenberg, Evan Suma

N1 - Funding Information: This work is sponsored by the U.S. Army Research Laboratory (ARL) under contract number W911NF-14-D-0005. Statements and opinions expressed and content included do not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred.

PY - 2018/7/2

Y1 - 2018/7/2

N2 - We present a dynamic omnidirectional texture synthesis (DOTS) approach for generating real-time virtual reality content captured using a consumer-grade RGB-D camera. Compared to a single fixed-viewpoint color map, view-dependent texture mapping (VDTM) techniques can reproduce finer detail and replicate dynamic lighting effects that become especially noticeable with head tracking in virtual reality. However, VDTM is very sensitive to errors such as missing data or inaccurate camera pose estimation, both of which are commonplace for objects captured using consumer-grade RGB-D cameras. To overcome these limitations, our proposed optimization can synthesize a high resolution view-dependent texture map for any virtual camera location. Synthetic textures are generated by uniformly sampling a spherical virtual camera set surrounding the virtual object, thereby enabling efficient real-time rendering for all potential viewing directions.

AB - We present a dynamic omnidirectional texture synthesis (DOTS) approach for generating real-time virtual reality content captured using a consumer-grade RGB-D camera. Compared to a single fixed-viewpoint color map, view-dependent texture mapping (VDTM) techniques can reproduce finer detail and replicate dynamic lighting effects that become especially noticeable with head tracking in virtual reality. However, VDTM is very sensitive to errors such as missing data or inaccurate camera pose estimation, both of which are commonplace for objects captured using consumer-grade RGB-D cameras. To overcome these limitations, our proposed optimization can synthesize a high resolution view-dependent texture map for any virtual camera location. Synthetic textures are generated by uniformly sampling a spherical virtual camera set surrounding the virtual object, thereby enabling efficient real-time rendering for all potential viewing directions.

KW - view-dependent texture mapping

KW - virtual content creation

KW - virtual reality

UR - http://www.scopus.com/inward/record.url?scp=85065519729&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85065519729&partnerID=8YFLogxK

U2 - 10.1109/ISMAR-Adjunct.2018.00040

DO - 10.1109/ISMAR-Adjunct.2018.00040

M3 - Conference contribution

AN - SCOPUS:85065519729

T3 - Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018

SP - 85

EP - 90

BT - Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 17th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018

Y2 - 16 October 2018 through 20 October 2018

ER -

Dynamic Omnidirectional Texture Synthesis for Photorealistic Virtual Content Creation

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access

OpenUrl availability

Other files and links

Fingerprint

Cite this