Discovering instructions for robust binary-level coverage criteria

Vaibhav Sharma; Taejoon Byun; Stephen McCamant; Sanjai Rayadurgam; Mats P.E. Heimdahl

doi:10.1145/3107091.3107092

Discovering instructions for robust binary-level coverage criteria

Vaibhav Sharma, Taejoon Byun, Stephen McCamant, Sanjai Rayadurgam, Mats P.E. Heimdahl

Computer Science and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Scopus citations

Abstract

Object-Branch Coverage (OBC) is often used to measure effective-ness of test suites, when source code is unavailable. The traditional OBC definition can be made more resilient to variations in compilers and the structure of generated code by creating more robust definitions. However finding which instructions should be included in each new definition is laborious, error-prone, and architecture-dependent. We automate the discovery of instructions to be included for an improved OBC definition on the X86 and ARM architectures. We discover all possible valid instructions by symbolically executing instruction decoders for X86 and ARM instructions. For each discovered instruction, we translate it to Vine IR, and check if the Vine IR translation satisfies the OBC definition. We verify the correctness of our tool by comparing its output with the X86 and ARM architecture manuals. Our automated instruction clas-sification facilitates development of more robust OBC definitions with better bug-finding ability and lesser sensitivity to compiler variations.

Original language	English (US)
Title of host publication	TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017
Editors	Tingting Yu, Darko Marinov
Publisher	Association for Computing Machinery, Inc
Pages	1-4
Number of pages	4
ISBN (Electronic)	9781450351126
DOIs	https://doi.org/10.1145/3107091.3107092
State	Published - Jul 13 2017
Event	1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, TECPS 2017 - Santa Barbara, United States Duration: Jul 13 2017 → …

Publication series

Name	TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017

Other

Other	1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, TECPS 2017
Country/Territory	United States
City	Santa Barbara
Period	7/13/17 → …

Bibliographical note

Funding Information:
This material is based upon work supported by the National Science Foundation under Grant Numbers 1563920 and 1514444. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. The authors wish to thank Qiuchen Yan for help with enumeration of X86 instructions.

Publisher Copyright:
© 2017 Association for Computing Machinery.

Keywords

Instruction classification
Object branch coverage

Access

10.1145/3107091.3107092

OpenUrl availability

Full text

Cite this

Sharma, V., Byun, T., McCamant, S., Rayadurgam, S., & Heimdahl, M. P. E. (2017). Discovering instructions for robust binary-level coverage criteria. In T. Yu, & D. Marinov (Eds.), TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017 (pp. 1-4). (TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017). Association for Computing Machinery, Inc. https://doi.org/10.1145/3107091.3107092

Discovering instructions for robust binary-level coverage criteria. / Sharma, Vaibhav; Byun, Taejoon; McCamant, Stephen et al.
TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017. ed. / Tingting Yu; Darko Marinov. Association for Computing Machinery, Inc, 2017. p. 1-4 (TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Sharma, V, Byun, T, McCamant, S , Rayadurgam, S & Heimdahl, MPE 2017, Discovering instructions for robust binary-level coverage criteria. in T Yu & D Marinov (eds), TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017. TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017, Association for Computing Machinery, Inc, pp. 1-4, 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, TECPS 2017, Santa Barbara, United States, 7/13/17. https://doi.org/10.1145/3107091.3107092

Sharma V, Byun T, McCamant S , Rayadurgam S , Heimdahl MPE. Discovering instructions for robust binary-level coverage criteria. In Yu T, Marinov D, editors, TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017. Association for Computing Machinery, Inc. 2017. p. 1-4. (TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017). doi: 10.1145/3107091.3107092

Sharma, Vaibhav ; Byun, Taejoon ; McCamant, Stephen et al. / Discovering instructions for robust binary-level coverage criteria. TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017. editor / Tingting Yu ; Darko Marinov. Association for Computing Machinery, Inc, 2017. pp. 1-4 (TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017).

@inproceedings{8fa4a2ec641c46b69553a61bfd2cbfc0,

title = "Discovering instructions for robust binary-level coverage criteria",

abstract = "Object-Branch Coverage (OBC) is often used to measure effective-ness of test suites, when source code is unavailable. The traditional OBC definition can be made more resilient to variations in compilers and the structure of generated code by creating more robust definitions. However finding which instructions should be included in each new definition is laborious, error-prone, and architecture-dependent. We automate the discovery of instructions to be included for an improved OBC definition on the X86 and ARM architectures. We discover all possible valid instructions by symbolically executing instruction decoders for X86 and ARM instructions. For each discovered instruction, we translate it to Vine IR, and check if the Vine IR translation satisfies the OBC definition. We verify the correctness of our tool by comparing its output with the X86 and ARM architecture manuals. Our automated instruction clas-sification facilitates development of more robust OBC definitions with better bug-finding ability and lesser sensitivity to compiler variations.",

keywords = "Instruction classification, Object branch coverage",

author = "Vaibhav Sharma and Taejoon Byun and Stephen McCamant and Sanjai Rayadurgam and Heimdahl, {Mats P.E.}",

note = "Funding Information: This material is based upon work supported by the National Science Foundation under Grant Numbers 1563920 and 1514444. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. The authors wish to thank Qiuchen Yan for help with enumeration of X86 instructions. Publisher Copyright: {\textcopyright} 2017 Association for Computing Machinery.; 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, TECPS 2017 ; Conference date: 13-07-2017",

year = "2017",

month = jul,

day = "13",

doi = "10.1145/3107091.3107092",

language = "English (US)",

series = "TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017",

publisher = "Association for Computing Machinery, Inc",

pages = "1--4",

editor = "Tingting Yu and Darko Marinov",

booktitle = "TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017",

}

TY - GEN

T1 - Discovering instructions for robust binary-level coverage criteria

AU - Sharma, Vaibhav

AU - Byun, Taejoon

AU - McCamant, Stephen

AU - Rayadurgam, Sanjai

AU - Heimdahl, Mats P.E.

N1 - Funding Information: This material is based upon work supported by the National Science Foundation under Grant Numbers 1563920 and 1514444. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. The authors wish to thank Qiuchen Yan for help with enumeration of X86 instructions. Publisher Copyright: © 2017 Association for Computing Machinery.

PY - 2017/7/13

Y1 - 2017/7/13

N2 - Object-Branch Coverage (OBC) is often used to measure effective-ness of test suites, when source code is unavailable. The traditional OBC definition can be made more resilient to variations in compilers and the structure of generated code by creating more robust definitions. However finding which instructions should be included in each new definition is laborious, error-prone, and architecture-dependent. We automate the discovery of instructions to be included for an improved OBC definition on the X86 and ARM architectures. We discover all possible valid instructions by symbolically executing instruction decoders for X86 and ARM instructions. For each discovered instruction, we translate it to Vine IR, and check if the Vine IR translation satisfies the OBC definition. We verify the correctness of our tool by comparing its output with the X86 and ARM architecture manuals. Our automated instruction clas-sification facilitates development of more robust OBC definitions with better bug-finding ability and lesser sensitivity to compiler variations.

AB - Object-Branch Coverage (OBC) is often used to measure effective-ness of test suites, when source code is unavailable. The traditional OBC definition can be made more resilient to variations in compilers and the structure of generated code by creating more robust definitions. However finding which instructions should be included in each new definition is laborious, error-prone, and architecture-dependent. We automate the discovery of instructions to be included for an improved OBC definition on the X86 and ARM architectures. We discover all possible valid instructions by symbolically executing instruction decoders for X86 and ARM instructions. For each discovered instruction, we translate it to Vine IR, and check if the Vine IR translation satisfies the OBC definition. We verify the correctness of our tool by comparing its output with the X86 and ARM architecture manuals. Our automated instruction clas-sification facilitates development of more robust OBC definitions with better bug-finding ability and lesser sensitivity to compiler variations.

KW - Instruction classification

KW - Object branch coverage

UR - http://www.scopus.com/inward/record.url?scp=85027677773&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85027677773&partnerID=8YFLogxK

U2 - 10.1145/3107091.3107092

DO - 10.1145/3107091.3107092

M3 - Conference contribution

AN - SCOPUS:85027677773

T3 - TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017

SP - 1

EP - 4

BT - TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017

A2 - Yu, Tingting

A2 - Marinov, Darko

PB - Association for Computing Machinery, Inc

T2 - 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, TECPS 2017

Y2 - 13 July 2017

ER -

Discovering instructions for robust binary-level coverage criteria

Abstract

Publication series

Other

Bibliographical note

Keywords

Access

OpenUrl availability

Other files and links

Fingerprint

Cite this