Statement on Guidance for the University of Pennsylvania Community on Use of Generative Artificial Intelligence

Be transparent about the use of AI.  Disclose when a work product was created wholly or partially using an AI tool and, if appropriate, how AI was used to create the work product.

The user of AI should endeavor to validate the accuracy of created content with trusted first party sources and monitor the reliability of that content. Users are accountable for their use of content created by AI and should be wary of misinformation or “hallucinations” by AI tools (e.g., citations to publications or source materials that do not exist or references that otherwise distort the truth).

When using AI, keep in mind that these tools are often trained on large, unmoderated bodies of text, such as text posted to the internet.  This can result in the production of biased and other unintended content. The ability to avoid such biased content is still in the early stages of development.

Most AI tools and services use input and data from users of the tool to train the model.  Additionally, existing tools may incorporate AI features in their service offerings. For this reason, users of AI should avoid sharing personal or sensitive data with the tool and should not  input moderate or high-risk Penn data as defined by the Penn Data Risk Classification, or intellectual property, without:

1. Careful consideration and understanding of the tool’s use of Penn data and the service provider’s stated rights to the data, including, but not limited to whether the service provider offers the option to opt-out of using customer’s data to train the AI;
2. A contract in place to protect Penn data; and
3. Review by Penn’s Privacy Office and consultation with the Office of Information Security as coordinated by Procurement when moderate or high-risk data is involved. 

Consultation with the Penn Center for Innovation, where intellectual property is involved.

Accordion content.It is not permissible under the Health Information Portability & Accountability Act (HIPAA) or Penn Medicine policy to share patient or research participant information in connection with open or public AI tools and services, such as ChatGPT. This is because, as currently configured, such open or public tools and services can use and share any data without regard to HIPAA restrictions and other protections. Therefore, individual patient data and patient data sets (even if deidentified) may not be exposed to open or public AI tools or services, absent institutional approval.  

When using AI to write computer code or when creating new technology that leverages AI it is important to be aware of the new kinds of cyberattacks that are being used against AI users.  Review the Office of Information Security guidance on these risks or consult with the Office of Information Security if in doubt. 

The rise of AI models has led to a significant increase in individuals and organizations scraping (i.e., copying) information posted on the internet for the purpose of training new AI models.  Be aware that any data posted publicly will likely be scraped and used in this way by third parties.  Similarly, while these practices are common, their legality and the potential consequences of these actions are currently being developed but remain unresolved at the time this guidance was issued. 

Members of the Penn community should adhere to established principles of respect for intellectual property, particularly copyrights when considering the creation of new data sets for training AI models.  Avoid uploading confidential and/or proprietary information to AI platforms prior to seeking patent or copyright protection, as doing so could jeopardize IP rights.

While automating tasks using AI may improve operational efficiency for University Business processes, oversight and review of the use of AI and verification of its outputs for these University business processes should be in place to ensure reliability, consistency, and accuracy.

• Use of AI should always be in alignment with Penn’s Principles of Responsible Conduct.
• As expectations may vary between classes and instructors, it is important for instructors to provide students with clear guidelines similar to the guidelines on collaboration, on the use of AI within coursework, and when and how the use of AI within a course should be cited. 
• Disclose to students when course materials have been created with the use of AI and when AI detection software will be used in the course.
• Be aware of how AI tools may impact assignments and exams in your discipline.  Penn’s Center for Teaching and Learning provides useful guidance in this space.

• All use of AI should be in line with Penn’s Code of Student Conduct and the Code of Academic Integrity.
• Educators may have requirements and guidance for citing the use of generative AI output and for attributing AI created content to the specific AI tool and parameters used.
• Individual courses may have different or more narrow guidance on the use of AI that should be adhered to within the context of that course.
• In the absence of other guidance, treat the use of AI as you would treat assistance from another person.  For example, this means if it is unacceptable to have another person substantially complete a task like writing an essay, it is also unacceptable to have AI to complete the task.
• Keep in mind that having access to data is not the same as having permission to scrape the data or use it to train an AI model.

• Consult with your department leadership and your discipline’s publishing standards to determine how the use of AI should be accounted for with regard to authorship in publications.
• Researchers should adhere to federal or international requirements on obtaining informed consent, and Institutional Review Board approvals should be obtained prior to exposing research participant data to AI tools.
• Caution should be adopted when research involves the examination of high-risk data, including Personally Identifiable Information (PII) and research participant health information (both identifiable and non-identifiable) exposed to AI.
• Intellectual property law may limit what AI can be copyrighted or patented depending on the specifics of their creation, and using data to train AI may implicate copyright laws and/or the patentability of an idea or discovery.  Researchers should avoid uploading confidential and/or proprietary information to AI platforms, at least prior to seeking patent or copyright protection, as doing so could jeopardize IP rights.  AI users should consult further with the Penn Center for Innovation for additional guidance, including determinations as to whether potential IP rights should be protected prior to the inclusion of data and findings in AI models.
• Researchers should reinforce with their mentees (undergraduate students, predoctoral students, and postdoctoral trainees) on the appropriate use of AI.

This AI Guidance is largely focused on large language models, such as ChatGPT and Google Bard; but these principles apply generally to other machine learning and artificial intelligence technologies.

This Guidance applies to Penn faculty, staff, undergraduate students, pre-doctoral students, and post-doctoral trainees.

The Offices of Information Security and University Privacy developed this Guidance in collaboration with the Office of the Vice Provost for Education, the Office of the Vice Provost for Faculty, the Office of the Senior Vice Provost for Research, the Office of Clinical Research, the Office of General Counsel, Procurement, and the Penn Center of Innovation.

Yes, Schools, Centers, and other departments may publish more specific guidance on the use of AI to address certain situations or use cases.     

• For questions about educational policies, please contact the Vice Provost for Education at provost-ed@upenn.edu.
• For questions regarding clinical research, please contact the Office of Clinical Research at lauraee@upenn.edu. 
• For questions about research use, please contact the Office of the Senior Vice Provost for Research at vpr@upenn.edu.  
• For questions about procurement, including contracting, sourcing, and supplier management, please contact Procurement Services at procure@upenn.edu.
• For questions regarding intellectual property and AI, contact the Penn Center for Innovation at pciinfo@pci.upenn.edu.
• For questions about privacy, please contact the University Privacy Office at privacy@upenn.edu.
• For questions about security, please contact the Office of Information Security at security@isc.upenn.edu.
• For legal questions related to this guidance document, contact the Office of General Counsel at Contact Us | OGC (upenn.edu).