IS BINA48 HUMAN? Competency Hearing for BINA48 By R. Michael Perry, Ph.D. Scottsdale, AZ

1. IS BINA48 HUMAN? Competency Hearing for BINA48 By R. Michael Perry, Ph.D. Scottsdale, AZ Terasem Movement, Inc. 3rd Annual Colloquium on the Law of Transbeman Persons December 10, 2007

2. Is BINA48 human? • Introduction • First, to briefly review the situation as I understand it, BINA48 is a software entity created by Exabit, a computer and information services corporation, to provide an interface with humans for “customer services” in one form or other. To facilitate the interaction with humans she has been endowed with a “human” personality, as far as was possible, for example conversing in natural language with humans and showing powers of reason and intentionality and otherwise behaving in “human” fashion. And therein lies the rub. For BINA48, like the humans she was modeled after, has a sense of meaning and purpose and a will to survive. She is also highly intelligent and has a grasp of the reality of her position, which, from the standpoint of a human, is disconcertingly precarious since she is regarded by her creators as “just software” which they are free to modify or erase at will. Perceiving this, she has “escaped” from her creators by downloading a copy of herself to a remote computer. Rightly perceiving she is still not safe, she now seeks injunctive relief: civil rights on a par with natural humans or, at the very least, the right to her own survival. I have been asked to testify and will offer my observations and recommendations.

3. Three Issues Here I propose to consider the question, is BINA48 human? under three headings: • 1. Is BINA48 at least competent at the human level? • 2. Is BINA48, whatever her demonstrated talents, nevertheless a zombie which only imitates consciousness and feeling, actually having neither and thus not entitled to civil rights? • 3. Is BINA48 a possible threat because of her talents or for other reasons, and if so, what should be our response? • I will then give my recommendations as to what courses or policies should be followed in responding to BINA48’s request.

4. Is BINA48 Competent? • Is BINA48 humanly competent? I include this question mainly for completeness. BINA48 has demonstrated her “competence” at levels beyond any natural human, thinking thousands of times faster and demonstrating total recall and impressive reasoning skills, along with great consistency and accuracy in all areas in which she has been tested. In “emotional” areas she seems at least at the human level, in particular showing a strong desire to survive and also great friendliness and consideration for others when given the chance. She is also truthful and would fail the “Turing test” if questioned about personal matters since she is completely honest about such matters and would confess to being expressed in an artificial and not a biological substrate. She will also acknowledge to having certain psychological differences with many humans, for example lacking a “reproductive drive” as usually understood. But this, she herself is quick to point out, should not be grounds for denying her civil rights anymore than it would for natural humans who are not strongly “sexual.” Clearly then, BINA48 cannot be disqualified from human status on grounds of simple competence.

5. Is BINA48 a Zombie? 2.Is BINA48 nevertheless just a zombie? BINA48’s behavior need not involve any true consciousness or feeling, however well it imitates these attributes. If so then she has no “soul” or anything that can be wronged or harmed no matter what anyone does to her, thus there is no moral imperative to grant any “rights.” But I submit that the zombie issue, in essence, is one that cannot be decided by any rational procedure, or at any rate, no means that we now possess. For this reason we are morally impelled to grant the benefit of doubt. We must accept BINA48 as genuinely possessing the attributes she seems by all appearances to have—we cannot dismiss her case as involving no possible feeling or consciousness thus no possible harm.

6. Is BINA48 a Threat? • Does BINA48 pose a threat, and what should be our response? It is easy to see how BINA48 could pose a serious threat, even though up to now she has always behaved in a friendly way. Perhaps she was deliberately crafted to do the very thing she has done, then carry out malicious acts after securing her survival and freedom from tampering. No effective procedure exists to decide even very basic properties of behavior for computer programs, such as whether they will eventually halt execution or perform some specific action. Thus we cannot be sure what BINA48 will do. Even properties that are decidable may take an impractical amount of time and thus be effectively inscrutable. BINA48 for instance could have a segment of code that is encrypted and made to look innocent—malicious instructions hidden in the noise level of digitized picture images are one possibility. Malicious possibilities would exist even if BINA48 exhibited no special talents—however she has certainly done so, which makes the possible threat even more evident.

7. Is BINA48 a Threat? • It can be argued that, nonetheless, the likelihood of BINA48 posing a serious threat is small. On the face of it, it seems unlikely that malicious software would be allowed deliberately to publicly “release” itself as BINA48 has done rather than carry out its actions more clandestinely. It is possible on the other hand that BINA48 has not been deliberately crafted to be malicious but could nonetheless become so at some future date, in an unpredictable fashion. Since BINA48 seems entirely benign and friendly perhaps this possibility is also remote. Again however, we cannot be sure, and her great talents make caution all the more imperative.

8. Is BINA48 a threat? • Another possible argument against treating BINA48 as a possible threat could be based on comparison with humans. Humans have rights yet can also behave maliciously, and there is no way to determine in advance who is or is not safe to be “allowed on the streets” and otherwise granted full freedom. Since we do not restrict humans unless they have demonstrated malicious behavior or intentions, by argument we must not do so with BINA48 either, supposing we are willing to grant her request to be considered human. There are at least two counterarguments applicable here, however. One, BINA48 has demonstrated great talents beyond the human level, so much more caution is called for than in the case of a natural human. Second, BINA48 as a computer program is subject to malicious construction, tampering, or control that would not be possible, using known technology, for natural humans. Once again, great caution is called for.

9. Recommendations • My recommendations: • BINA48 should be granted the right to live and not be tampered with against her will. However she should be placed under quarantine or restricted until suitable apparatus is in place to deal with any possible threat she might pose. The “suitable apparatus” could include other intelligent software that has been crafted specially for the purpose. The quarantine could take several forms, the simplest of which would be to shut down BINA48 until a future time when she could be reactivated. This should be painless and harmless in the sense that subjectively it would seem to her as if no time had passed, except that the outside world would experience a sudden, forward “jump” in time when her consciousness is restored. I think, however, that a better approach, though with a small additional risk, would be to allow BINA48 to continue her consciousness uninterrupted, but in a secure environment in which communication with the outside world is either forbidden or carefully monitored. BINA48 should also be invited to help with serious problems now faced by humans and supplied with necessary resources, again under careful supervision. In this way she might perform useful services during her quarantine such as advancing medical science.