Melbourne University Law Review
|
|
Home
| Databases
| WorldLII
| Search
| Feedback
Melbourne University Law Review |
|
Rita Matulionyte[1]*
The increasing use of automated decision-making systems (‘ADMS’) in the government sector causes various problems. One of them is the lack of transparency around government algorithms, especially when they are developed by third parties. This article explores
how proprietary interests surrounding government automation technologies, such as
trade secrets, affect transparency around ADMS and what measures could be taken to preserve — and increase — government transparency in the age of automation. The article focuses on New South Wales as a case study and explores gaps within the Government Information (Public Access) Act 2009 (NSW) (‘GIPA Act’) in ensuring transparency in the government automation context. It recommends that current legislative gaps be addressed by: (1) narrowing down the protection for trade secrets under the GIPA Act;
(2) improving access to government information held by third parties (eg government contractors); and (3) ensuring that the government leverages its procurement powers to acquire information that is essential for public transparency purposes.
Federal and state governments in Australia have been heavily investing in the digitisation of government services. The Commonwealth government’s digital transformation strategy sets an ambitious vision for the Australian government to become ‘one of the top three digital governments in the world by 2025’,[1]
with a $1.2 billion investment devoted to the implementation of this vision in 2021–22 alone.[2] Similarly, the New South Wales (‘NSW’) government has adopted the Beyond Digital strategy,[3] which focuses inter alia on emerging technologies, and the Artificial Intelligence Strategy which sees artificial intelligence (‘AI’) as a tool that could ‘improve service delivery and government decision-making’.[4] AI, and automated decision-making systems (‘ADMS’) more broadly, are already used in numerous federal and state government services. For instance, the federal government uses algorithms to analyse relevant medical data from the Australian ‘My Health Record’ system.[5] Algorithms are also implemented in the automated border control system, ‘SmartGate’, to verify the identity of travellers entering and leaving the country.[6] ADMS were also used by Centrelink to calculate government overpayments (ie Robodebt).[7] Examples of the NSW government’s utilisation of ADMS include: Transport for NSW’s use of smart road safety cameras;[8] Revenue NSW’s AI tool ‘to identify and support customers facing hardship’;[9] and NSW Health’s trials of an
AI-enabled sepsis identification tool in NSW hospitals.[10]
While emerging technologies have the potential to make government services more effective, fair and accessible,[11] AI and other ADMS cause numerous problems;[12] one of which is the lack of transparency around the use of these new technologies. A report by the NSW Ombudsman, The New Machinery of Government: Using Technology in Administrative Decision-Making, concluded that there is ‘an almost complete lack of transparency about [the use of government machine technology]’.[13] Recent case law also demonstrates the lack of transparency around automated decision-making in NSW. In cases such as O’Brien v Secretary, Department of Communities and Justice (NSW) (‘O’Brien’)[14] and Ireland v Central Coast Council (‘Ireland’),[15] which will be discussed below, individuals were rejected access to certain information about ADMS employed by NSW agencies, despite these systems directly affecting their rights or legitimate interests.[16]
Transparency around ADMS used by governments is generally important for ensuring government accountability and trust.[17] Transparency is an essential attribute of an open and democratic government,[18] and is a practical realisation of the human right to information that is implemented in international human rights treaties.[19] In the automated government context, transparency is especially important due to the significant risks associated with the use of new and emerging government technologies (eg AI technologies). These risks include a lack of regulation, insufficient accuracy, possible bias, and privacy, data security and data breach risks, as well as more general ethical concerns.[20] Keeping in mind that these new technologies often lack established quality assurance and independent oversight frameworks, public transparency becomes especially important in enabling visibility and public scrutiny of these technologies and facilitating government accountability.[21] In addition, the information asymmetry that is increasingly visible in the digital and automated government context leads to calls for increased transparency. Namely, as digital governments hold increasing amounts of information about citizens and possess automated tools to efficiently analyse this data, there is a need to improve individuals’ access to government information to address this growing asymmetry.[22] Empirical studies have confirmed public interest in more transparency around automated government technologies.[23]
While Australia’s federal and state governments are generally committed to transparency,[24] and in the AI and ADMS context in particular,[25] the implementation of this principle in relation to new and emerging government technologies has proven to be challenging.[26] One of the reasons behind the lack of transparency is commercially confidential information (or trade secrets)[27] that often results from government and private sector partnerships that are formed to develop and implement government automation tools. When third parties (eg government contractors) are involved in the development, deployment or maintenance of government ADMS, much of the information is either held by contractors or, even if held by the government, kept confidential due to its commercial nature and cannot be easily publicly accessed.[28]
The purpose of this article is thus to examine the reasons behind the lack of transparency around government ADMS and possible solutions to improve ADMS transparency. This article will focus particularly on how trade secrets affect the transparency of government ADMS and how the current legislative frameworks could be improved to address the transparency challenges caused by such proprietary information.
The article will use NSW as a case study to examine the impact of trade secrets on the transparency of government ADMS and possible law reform pathways. The NSW government is one of the most advanced state governments in Australia, both in terms of its digitisation of government activities and its open government policies.[29] The recommendations considered in the context of NSW law will be relevant in addressing comparable trade secret challenges in other Australian jurisdictions[30] and internationally.[31]
Part II of this article will briefly revisit the concept of transparency and examine transparency challenges in relation to ADMS in NSW. It will demonstrate that despite the commitment to transparency in general and in relation to ADMS more specifically, there is still a significant lack of publicly available information about automated tools used by the NSW government. Part III will focus on trade secrets as one of the barriers to ensuring transparency around government ADMS. By delving into the Government Information (Public Access) Act 2009 (NSW) (‘GIPA Act’), Part III will demonstrate legal gaps that impede public access to government information about ADMS in which private parties have proprietary interests, both when such information is held by government and when it is held by government contractors. Part IV develops proposals on how current legislative gaps in NSW could be addressed to ensure that the right to information is retained in the age of government automation, despite increasing government and private sector partnerships and the commercial secrecy that they generate.
Transparency is a multifaceted and evolving concept with numerous proposed definitions in the literature.[32] In the AI and ADMS context, various interrelated concepts have been used — transparency, explainability, interpretability and contestability — without a clear consensus on what each of them mean and how they can be delineated. For example, under Australia’s AI Ethics Principles, AI ‘transparency and explainability’ is defined as a requirement for ‘responsible disclosure so people can understand when they are being significantly impacted by AI, and can find out when an AI system is engaging with them’.[33] In contrast, NSW’s AI Ethical Policy refers to ‘transparency’ only, which is understood as a requirement that ‘[t]he community should be engaged on the objectives of AI projects and insights into data use and methodology should be made publicly available unless there is an overriding public interest in not doing so’.[34] The NSW policy further emphasises the capability of ‘transparency’ to ensure that there are review mechanisms through which ‘citizens can question and challenge AI-based outcomes’.[35]
It is clear that transparency is important to different stakeholders for different reasons.[36] As summarised in Australia’s AI Ethics Principles, transparency is important for the following group of stakeholders for the following reasons:
[F]or users, what the system is doing and why; for creators, including those undertaking the validation and certification of AI, the systems’ processes and input data; for those deploying and operating the system, to understand processes and input data; for an accident investigator, if accidents occur; for regulators in the context of investigations; for those in the legal process, to inform evidence and decision‐making; [and] for the public, to build confidence in the technology ...[37]
This article focuses on transparency around government ADMS that are directed to the public (‘public transparency’) and leaves other types of transparency (eg transparency for auditors, incident investigators and supervisory authorities) outside the scope of this article.[38] ‘Public transparency’ is defined broadly as the availability of information about automated systems that the government is using — their development processes (including training, validation and testing), use purposes, functioning and outcomes — with the purpose of ensuring government accountability, contestability of decisions and public trust.[39]
It should be acknowledged that despite its importance and potential, transparency alone is not capable of ensuring the quality of government decisions and will not in all cases result in government accountability in
the automated decision-making context. Limitations of the government transparency and open government concepts have been extensively discussed in the literature,[40] including transparency laws that are too abstract, exceptions to government’s transparency obligations that are too broad, transparency’s inability to guarantee government accountability and weak judicial enforcement of transparency laws.[41] However, these and other limitations surrounding the government transparency concept do not negate its overall importance in ensuring open and accountable government. Rather, they call for ways to improve how government transparency is ensured, which is the ultimate intention of this article.
There is no doubt that NSW is committed to transparency generally and in relation to digital and automated government specifically. The GIPA Act, as the main piece of legislation in this area, has an object to make government information publicly accessible ‘[i]n order to maintain and advance a system of responsible and representative democratic Government that is open, accountable, fair and effective’.[42] In summary, the GIPA Act requires twofold transparency from NSW agencies. First, NSW agencies are required to proactively release certain information (‘proactive transparency’). This includes ‘open access’ information,[43] such as an agency’s information guide, policy documents, disclosure log of access applications, and other information listed under s 18 of the GIPA Act, unless there is an ‘overriding public interest’ against such release.[44] The NSW Information and Privacy Commission (‘NSW IPC’) has explained that in the ADMS context, ‘open access’ provisions require agencies to identify datasets that agencies hold, provide ‘inventories of machine enhanced decision-making systems’, and ensure that documents and information that support ‘digital government and automated decision-making [have] been developed’ and are ‘available as open access information’.[45]
In addition to proactive transparency, agencies are required to provide information that is requested by members of the public, either via informal or formal access requests (‘reactive transparency’).[46] If a formal information access request is lodged by a member of the public or other interested stakeholder (eg a business), government agencies have an obligation to release the requested information unless there is an overriding public interest against such release.[47]
The GIPA Act framework is further supported by several other NSW policies. For instance, the NSW IPC’s Charter for Public Participation: A Guide To Assist Agencies and Promote Citizen Engagement ‘provides a practical road map for embedding the functions of the GIPA Act’ into the activities of NSW agencies.[48] In the digital and automated government context, the NSW Beyond Digital strategy[49] requires data to be open, collected and shared with the public, while respecting the confidentiality and privacy of individuals.[50] The NSW IPC’s ‘Open Government, Open Data & Public Participation’ policy further
encourages the proactive public release of government information by agencies in ways that are respectful of data sharing safeguards, as well as providing information, advice and assistance to agencies and members of the public on access to government information.[51]
Additionally, transparency is implemented as one of the principles under NSW’s AI Ethical Policy, with the aim of facilitating the contestability and reviewability of government decisions.[52]
NSW government reports indicate that the public is increasingly interested in information held by the government, with public requests for government information more than doubling over the last decade.[53] Community Attitudes surveys conducted by the NSW government also suggest that ‘over 85% of members of the public value their right to access information’.[54] Despite this, government transparency, especially around automated decision-making, remains a challenging field.
First, despite proactive transparency requirements set under the GIPA Act (eg requirements related to open access information) and the NSW IPC guidance on how these rules apply to government ADMS,[55] limited information about government ADMS is currently being made available as open access information (ie on agencies’ websites). Notably, the NSW Ombudsman found a significant lack of publicly available information on what ADMS NSW agencies are using, for what purposes and how they are governed.[56] According to the NSW IPC, the rate of compliance with open access requirements across all agencies is relatively high (72% in 2020).[57] However, the NSW IPC data do not reflect the public availability of information about ADMS used in NSW; the NSW IPC’s GIPA Act compliance reviews are of a general nature and do not examine the type or depth of information made available in open access. Thus, it is not clear from NSW IPC reports how much information about digital and automated systems is available as open access information.
Some further evidence suggests that agencies do not proactively disclose information about the ADMS they use. For instance, it has been previously reported that Revenue NSW, a part of the Department of Customer Service (NSW), had used automated technology to issue garnishee orders without human oversight, which was heavily criticised.[58] More recently, the Department has introduced another automated tool that uses AI to ‘identify and support customers facing hardship’.[59] Despite searching the Department’s information guide, no information was found about either of these automated systems.[60] The guide neither contained ‘inventories of machine enhanced decision-making systems’, nor referred to documents that support digital government or automated decision-making, as recommended by the NSW IPC.[61]
Similarly, while reactive transparency of NSW agencies has remained stable over the years,[62] significant challenges remain. Data shows that a significant amount of information requested by the public from NSW agencies is not being disclosed or is only partially disclosed. For instance, during 2021–22, 70% of access requests were granted in full or in part, with the remaining 30% of applications not satisfied for various reasons.[63] While the NSW Agency GIPA Dashboard does not have specific statistics relating to digital and automated government services,[64] previous NSW IPC studies suggest that similar rates for disclosure of information apply to information related to digital government (including automated decision-making).[65]
Most concerning are situations where requested information was not released because it was not held by the government but, instead, by government contractors. Several recent cases demonstrate this problem. For instance, in O’Brien, the applicant sought information from the Department of Communities and Justice (NSW) about rental subsidies; namely, policies and other documents regulating the automatic calculation of base rent and rental subsidies as well as the algorithm, software specification, source code and test suites related to the rental subsidies calculation tool.[66] The Department determined that it did not hold the algorithm, software specification, source code or test suites sought by the applicant, as this information was held by the third party/contractor Northgate Public Services (‘NPS’) that developed the software.[67] While the NSW Information and Privacy Commissioner found the Department’s decision unjustified,[68] the NSW Civil and Administrative Tribunal affirmed the Department’s decision, agreeing that the requested information was not held by the government.[69] The Tribunal also found that the contract between the Department and NPS (signed before the enactment of the GIPA Act) did not provide the government with a right to access information held by the third party.[70] The Tribunal held that, even if such a right to access information existed under the contract, it was doubtful that the contractor would have a duty to provide access to information such as algorithms, source code and related data since disclosing this information would lead to commercial disadvantage for the contractor.[71]
In another case, Ireland, the applicant, Mr Ireland, sought access to documents relating to flood depth level reports about his property, one of which related to engineering calculations produced by an aerial software program that was developed by a third party and used by the Central Coast Council to measure the flood depths.[72] The Council determined that it did not hold the requested information and rejected the request.[73] In contrast to O’Brien, the NSW Civil and Administrative Tribunal found that the Council’s decision in Ireland did not provide sufficient evidence that the Council did not hold the information and remitted the case back to the Council.[74] It is to be seen whether the final outcome will be similar to O’Brien.
Overall, despite the general commitment to transparency in NSW, there appears to be challenges in ensuring both proactive and reactive transparency around government ADMS. This is concerning considering the scale of investment in, and the increasing adoption of, such systems by the NSW government, as well as the possible risks such systems present for human rights and legitimate interests.[75] There is therefore a need to understand the reasons underlying the lack of transparency around government ADMS and the possible ways of addressing them.
Insufficient transparency around government ADMS might be caused by different reasons.[76] For instance, information that stakeholders are interested in may not have been documented and thus nobody holds it;[77] information may not be held by the government but by third parties that develop or operate the system;[78] it may be too technically complex or too resource intensive to locate the requested information;[79] or information might not be disclosed to the public due to competing public interests.[80] This Part will explore how public-private partnerships established for government automation purposes, and the proprietary information and trade secrets resulting from such partnerships, affect the transparency of government ADMS, and whether the current NSW GIPA Act is sufficiently equipped to deal with these emerging issues.
As already indicated in the introduction, trade secrets — or more broadly, sensitive business information of third parties — is one of the reasons behind the lack of transparency around government ADMS. Their importance in the government ADMS transparency context has been identified by government institutions, by commentators and in court decisions.[81]
For instance, the NSW Information and Privacy Commissioner has highlighted problems with accessing digital government information due to ‘contractual relationships and claims of commercial-in-confidence’.[82] In relation to the NSW Digital Restart Fund that finances government digitisation projects, the NSW Information and Privacy Commissioner warned of the risk that government agencies may not be able to access information held by third parties who help develop government automation tools.[83] In particular, third-party vendors or contractors, such as ‘providers of new platforms, software and/or cloud storage solutions ... may hold government information but are not covered by the GIPA Act’.[84] This means that they cannot be required to provide information that was traditionally considered government information and therefore had to be supplied to the public either proactively or reactively.
The abovementioned cases of O’Brien and Ireland illustrate how public-private partnerships generally, and commercial interests more specifically, can create challenges in ensuring transparency — ie by denying public access to information held by government contractors. Trade secrets might inhibit not only reactive transparency, as exemplified in the above cases, but also proactive transparency. Namely, agencies might decide not to proactively release certain information about ADMS due to trade secrets. This is already seen in relation to government contracts, where commercially sensitive information is hidden or deleted before the contract is published on a public registry.[85] Numerous commentators have highlighted the negative role of trade secrets in ensuring transparency around government ADMS,[86] as well as more general problems related to public-private partnerships in the government automation context.[87]
While trade secrets are not a new problem, the role of third parties in the automated government context and accordingly, the impacts of their trade secrets, are likely to increase in the future. Governments, including the NSW government, are increasingly digitising and automating their services.[88] Due to the specialised technical skills required in such projects, governments often have no choice but to participate in partnerships with commercial partners and outsource the development, operation and maintenance of automated tools on which they rely in delivering their services.[89] Among the various problems related to such partnerships,[90] private trade secrets stand out as one of the most significant challenges.[91] The developers of information technologies, including government contractors, increasingly rely on trade secrets to protect their intellectual property (‘IP’) in automated tools. One of the reasons for this is that other IP rights (such as copyright or patents) are often not available for complex algorithms or only protect AI models to a limited extent.[92] Trade secrets are thus becoming a dominant type of IP right that ADMS developers use to protect their commercial interests.[93] All of this suggests that the problem of access to government information in which third parties have commercial interests is likely to increase in the future.
Our next question is thus whether current NSW laws are sufficiently equipped to meet the challenge of ensuring public access to information that is important for transparency and accountability around government ADMS, even in the context of public-private partnerships. Since the GIPA Act treats the disclosure of information held by the government[94] and the disclosure of information held by third parties (eg government contractors)[95] differently, I will examine these two scenarios separately.
To start with, the GIPA Act generally applies to ‘government information’ that is defined as ‘information contained in a record held by an agency’,[96] which means that the government can only disclose information that they ‘hold’, and not information held by third parties. When a government agency intends to make this government information available to the public, it is required to assess whether there is no ‘overriding public interest against the disclosure’.[97] In the table appended to s 14, the GIPA Act lists seven categories of such information, or ‘exceptions’ to the government disclosure duties;[98] one of which is ‘[b]usiness interests of agencies and other persons’.[99] This ‘business interests’ exception covers the disclosure of information that would: (a) ‘undermine competitive neutrality’ of the agency or place it at a competitive advantage or disadvantage in a market;[100] (b) ‘reveal commercial-in-confidence provisions of a government contract’;[101] (c) ‘diminish the competitive commercial value of any information to any person’;[102] or (d) ‘prejudice any person’s legitimate business, commercial, professional or financial interests’.[103] Trade secrets and other business interests are likely to fall under items 4(b), (c) or (d).[104]
Importantly, this ‘business interests’ exception, or any other exception under s 14 of the GIPA Act, does not apply automatically. If an agency is requested to provide information that falls under the exception, it is required to consult with the third party that could be possibly affected by the disclosure of the information[105] and balance the two types of public interests:[106] the public interests favouring disclosure of government information[107] and the public interests in protecting information covered by the exception. This means that trade secrets or other business information are not automatically given priority and need to be weighed against the public interest in disclosing information held by the government. The GIPA Act establishes a general presumption in favour of disclosure of government information.[108]
The ‘business interests’ exception, however, is unreasonably broad, which is especially problematic in the government ADMS context, where public-private partnerships and the resulting private interests are playing an increasing role. There are three specific problems related to the current wording of the ‘business interests’ exception under the GIPA Act.
The first problem with the ‘business interests’ exception under the GIPA Act is the Act’s extremely broad definition of ‘commercial-in-confidence’ information under s 4(1) — or, broadly speaking, trade secrets[109] — which allows the government to reject public disclosure of a wide range of business-related information.[110] The GIPA Act defines ‘commercial-in-confidence provisions’ as any provisions of a contract that disclose a contractor’s financing arrangements, cost structure, profit margins or full base case financial model, as well as ‘any intellectual property in which the contractor has an interest’ or ‘any matter the disclosure of which would place the contractor at a substantial commercial disadvantage in relation to other contractors or potential contractors, whether at present or in the future’.[111]
This definition of commercial confidential information seems to be broader than information protected under the breach of confidence doctrine, which protects confidential information in various commercial and non-commercial settings.[112] Under the breach of confidence doctrine, the information must have ‘the necessary quality of confidence about it’, it must have been ‘imparted in circumstances importing an obligation of confidence’ and there must be an ‘unauthorised use of that information’.[113] In contrast, the definition of ‘commercial-in-confidence provisions’ under the GIPA Act does not contain an explicit ‘confidentiality’ requirement; its definition seems to cover any of the categories of information listed under s 14, be it confidential or not. Apart from trade secrets, ‘any intellectual property’ can include copyright, patents, trademarks, plant variety rights etc. Registered IP rights (eg patents, trademarks and plant variety rights) cannot be confidential as information about them is required to be published in public registers.[114] Unregistered rights such as copyright[115] are not confidential unless the party decides to keep copyrighted content confidential.
Further, the definition of ‘commercial-in-confidence’ includes information that would place a contractor at a ‘substantial commercial disadvantage ... at present or in the future’. Since this definition does not require that information is subject to confidentiality requirements, it is generally broad and was also broadly interpreted by the NSW judiciary.[116] According to this definition, ‘commercial-in-confidence’ information might cover not only financial information that legitimately deserves protection (eg financial arrangements and profit margins)[117] but also information that is essential for quality assurance purposes (eg algorithmic parameters, training, external validation, and assessment or audit information). The disclosure of this information, especially if it indicates issues related to the quality of the technology, is likely to cause ‘substantial commercial disadvantage’ to the third party. At the same time, the public has a legitimate interest in accessing such information in order to assess the quality of government technology, especially if it makes or impacts decisions directly affecting the interests of individuals.
In comparison, trade secret laws in other jurisdictions have a comparable ‘independent commercial value’ requirement. Namely, both United States (‘US’) and European Union (‘EU’) trade secret laws, in addition to a secrecy element, require that the information claimed to be a trade secret be of value and that the value of this information derives, at least in part, from the fact of secrecy.[118] However, this ‘independent commercial value’ requirement is defined in a narrow way. In order to meet this requirement, it is not sufficient to prove that the information owner ‘has gone to time, trouble, and expense’ to collect information.[119] Rather, the essential inquiry is whether ‘the alleged trade secret, as opposed to the AI system as a whole, gives its holder a non-trivial advantage over competitors’.[120] Sandeen and Aplin argue that this requirement, if properly applied, would result in the absence of trade secret protection for much of the information related to AI systems.[121] For instance, information relevant for quality assurance and accountability purposes, such as validation, external assessment or auditing results, is unlikely to give competitors a non-trivial advantage, as they would not be able to use it for developing their own products or businesses. In contrast, under the current wording of the GIPA Act, this information is likely to be covered by the broad ‘commercial-in-confidence’ definition and its public disclosure is likely to be denied.
In addition to confidential contractual information (s 14 item 4(b) discussed above), the GIPA Act ‘business interests’ exception covers a range of other information related to third parties’ interests; namely, information the release of which would ‘diminish the competitive commercial value of any information to any person’,[122] and ‘prejudice any person’s legitimate business, commercial, professional or financial interests’.[123] In contrast to item 4(b), items 4(c)–(d) cover not only contractual provisions, but also any other commercially relevant information held by the government.
The broad and unqualified nature of business information protected under items 4(c)–(d) is concerning. First, there are no thresholds or limitations to these exceptions. In contrast to item 4(b), there is no requirement that the competitive value of information be ‘significantly’ diminished or that the legitimate commercial or professional interests be ‘substantially’ prejudiced upon its disclosure. This means that any diminution of value or any prejudice to business interests of third parties should be taken into account when determining whether the information can be disclosed to the public.
Second, information protected by these provisions does not necessarily need to be confidential at all.[124] This raises a question as to why the GIPA Act should deny disclosure of information that is not confidential. If information had significant commercial value and advantage, it is reasonable to expect that their owners would keep it confidential, allowing for it to be protected under the breach of confidence doctrine. If it is not treated as confidential by an interested third party, it is questionable whether its commercial value is so significant that it deserves protection under the GIPA Act. Under the general breach of confidence law, even commercially valuable information would generally not be legally protected from disclosure unless the organisation made efforts to keep it confidential.[125] It is thus questionable why the GIPA Act extends protection to information that is not kept confidential by an organisation and that is not protected by general laws protecting confidential information, such as a breach of confidence doctrine.
Finally, it is questionable whether the public interest balancing test, as defined under ss 13–16 of the GIPA Act, adequately enables agencies to weigh interests in favour of and against disclosure, especially when dealing with the ‘business interests’ exception. This test can be praised for its flexibility and its usefulness when agencies need to disclose certain non-confidential commercial information in which there is a strong public interest. At the same time, it is questionable whether the public interest balancing test is of help when dealing with confidential commercial information — ie trade secrets.[126] It is difficult to imagine that an agency would ever disclose confidential information and expose themselves to risk of liability under the breach of confidence doctrine, even if there is a strong interest in disclosing this information to the public. For instance, even if an individual needs access to some algorithmic information in order to challenge a decision that affects them, agencies are very unlikely to disclose this commercial information if it is confidential for several reasons.
First, the GIPA Act itself seems to favour trade secrets over public disclosure in some cases. For instance, under s 32(1) of the GIPA Act, while certain government contracts with third parties should be published in the government contracts register, ‘the commercial-in-confidence provisions of a contract’ do not need to be included in the register. Here, preference is given to trade secrets and a balancing of public interests does not apply at all. Second, Australian law provides broad protection to confidential information, with very limited exceptions to such protection.[127] Government agencies, being risk-averse organisations, are unlikely to subject themselves to legal risk by disclosing information protected by law.[128] Further, O’Brien shows that the NSW judiciary may have a tendency to strongly protect not only trade secrets but all IP rights around ADMS, regardless of their actual secrecy status. In O’Brien, the NSW Civil and Administrative Tribunal suggested that information protected by any IP right does not need to be publicly disclosed.[129] The Tribunal gave a clear preference to IP rights, without any significant effort to consider whether IP interests outweighed the interest of an individual to understand exactly how the ADMS decision that impacted them was made.[130]
The second situation that needs to be discussed separately is where government information is held not by the government but by a government contractor. While the GIPA Act provisions specifically regulating the disclosure of information held by government contractors should generally be welcomed, they face a few gaps that prevent their effective implementation in the government ADMS context.
The public disclosure of information held by government contractors is currently addressed via s 121(1) of the GIPA Act (‘immediate access’ provision). First, the section defines situations in which it applies, namely, where ‘[a]n agency ... enters into a contract (a government contract) with a private sector entity (the contractor) under which the contractor is to provide services to the public on behalf of the agency’. Section 121(1) then suggests that, in cases of such contracts, the agency ‘must ensure that the contract provides for the agency to have an immediate right of access’ to the following information held by the contractor: (a) ‘information that relates directly to the performance of the services by the contractor’;[131] (b) ‘information collected by the contractor from members of the public to whom it provides’ the services;[132] and (c) ‘information received by the contractor from the agency to enable it to provide the services’.[133] Lastly, s 121(2) provides a list of information that is excluded from the disclosure requirement. This information includes: (a) information about a ‘contractor’s financing arrangements, financial modelling, cost structure or profit margins’;[134] (b) ‘information that the contractor is prohibited from disclosing’ under law;[135] and (c) information that ‘could reasonably be expected to place the contractor at a substantial commercial disadvantage in relation to the agency, whether at present or in the future’.[136]
The NSW IPC has developed a few general guidelines for agencies
and contractors on how to follow these requirements in practice,[137] as well
as guidelines targeting digital government projects.[138] Despite this, s 121 is problematic in the ADMS context in a few respects.
First, s 121(1) of the GIPA Act applies only where the government contractor ‘provide[s] services to the public on behalf of the agency’. In O’Brien, the Department of Communities and Justice (NSW) argued that the government contractor that held the requested information, NPS, merely designed software used by the Department to provide services.[139] Since NPS was not directly providing services to the public, s 121 arguably did not apply. Despite the NSW Information and Privacy Commissioner’s suggestions to interpret the section in a broader manner,[140] the Tribunal agreed with the Department’s conclusion that the contractor did not provide direct services to the public.[141]
This decision illustrates that the wording of s 121 and its current construction are too narrow and incompatible with the goals of the GIPA Act. Increasingly, government contractors are not requested to provide services to the public on behalf of the government but, rather, to assist them in the commencement of such services by developing tools to help make decisions, providing access to databases and smart platforms etc. The types of government and private sector partnerships that exist have changed significantly since the commencement of the GIPA Act in 2010.[142] As a result of various partnerships, government contractors could hold important information in which the public may have a legitimate interest. For instance, when a contractor develops a system that automates a government decision or a part of the decision-making process, members of the public have a legitimate interest in knowing exactly how the system functions, and in being able to scrutinise and check its accuracy and reliability. This is a prerequisite for the ability to challenge decisions made with the help of ADMS, to protect public interests and hold the government accountable. Section 121 focuses only on old-type contracts on the outsourcing of government services and excludes new types of government and private sector partnerships. The current wording of s 121 and its narrow construction thus have the potential to significantly reduce access to government information, decrease government transparency and accountability, cause harm to the interests of individuals and businesses, and eventually negatively affect trust in government services.
The second problem related to s 121 is that access to government information held by government contractors needs to — and can only be — established via a government contract. The statute does not set a statutory duty for government contractors to provide access to government information that they hold. In O’Brien, the government contract with the third party, NPS, did not contain provisions providing government access to any information.[143] Importantly, this contract was signed before the enactment of the GIPA Act and thus the Tribunal considered that such a duty cannot be retrospectively introduced into the contract.[144] However, even in relation to contracts signed after the enactment of the GIPA Act, there is no guarantee that all government contracts have such an ‘immediate access’ provision. If such contractual information duties are omitted from the government contract (by error or otherwise) — and since there is no statutory duty to provide information on government contractors — governments would not be contractually able to request any information from third-party contractors, even if this information is essential for transparency, decision contestability or government accountability purposes.
Third, the categories of information to which a contract should establish ‘immediate access’, as listed under s 121(1) of the GIPA Act, are either narrow or vague. It is quite obvious that a contractor should provide access to information that they receive from members of the public and from the agency. These provisions — ss 121(1)(b)–(c) — are thus justified but they are narrow in scope. The remaining s 121(1)(a) refers to information that ‘relates directly to the performance of the services’. The exact information that is supposed to be covered by this provision remains unclear. If this exact wording was replicated in a contract,[145] would it cover information generated by the contractor, such as algorithm structure, parameters, training, validation data or results? Would it cover independent auditing results and other external quality assurance information about the ADMS? Due to the vagueness of the provision, the contractor could argue that information about the algorithm and its validation and testing does not ‘relate ... directly to the performance of the services’,[146] but rather sets technical parameters behind the tool that is used in government service provision. Thus, returning to O’Brien, even if such a vague information duty existed in the contract between the NSW government and the contractor, it is doubtful that it would have covered the information that was requested by the affected individual (eg algorithm, software specification, source code and test suites).
The fourth issue related to s 121 is the broad exceptions to the ‘immediate access’ provisions. Most problematically, a government contract is not required to provide for the agency to have an immediate right of access to information that would place a contractor in a ‘substantial commercial disadvantage in relation to the agency, whether at present or in the future’.[147]
The concept of ‘substantial commercial disadvantage’, which was already discussed above in relation to the ‘business interests’ exception,[148] lacks legal certainty and predictability, and has been interpreted by the NSW Civil and Administrative Tribunal in an unreasonably broad manner. In O’Brien, the Tribunal determined that disclosure of the information requested by the applicant, namely the algorithm, source code and test suites
could reasonably be expected to place the contractor (NPS) at a substantial commercial disadvantage in relation to the agency (DCJ) whether at present or in the future. Essentially such disclosure could lead to the position that DCJ no longer needed to engage in the specialist services of NPS to facilitate DCJ carrying out its business and discharging their statutory functions. On this analysis s 121(2)(c) is clearly enlivened and would operate to prevent the requirement for access to the information by DCJ, to meet the criteria that the information is held.[149]
The Tribunal agreed with the submission of the Department of Communities and Justice (NSW) that if released,
the information could then be used by the Respondent to replicate and/or create the same or a similar software product ‘in house’ or use the source code and algorithms (intellectual property) to request another service provider to provide the service at a lower cost and thus place the contract between NPS and the Respondent in jeopardy.[150]
This is an unreasonable argument and more specifically, appears to demonstrate the Tribunal’s misunderstanding of IP rights and their scope. The Department’s access to the algorithm, source code or training datasets does not itself allow the Department to share it with the public, replicate it or create the same software product. Much of this information, especially the source code, remains protected under copyright law. Thus, access to subject matter protected by IP laws (more specifically, copyright) does not enable the government agency to reproduce it or make it available to another service provider and request them ‘to provide the service at a lower cost’. Copying and publicly communicating work protected by copyright would result in copyright infringement.[151]
Indeed, it is very difficult — if not impossible — to think of ways that the disclosure of an algorithm and other technical information could put the contractor at a substantial commercial disadvantage in relation to the agency. One of the consequences might be that the agency learns of the algorithm’s flaws from this technical information and thus decides to discontinue the contract. For example, the agency may learn that the algorithm or the model is of low quality, that it is based on incorrect formulas or assumptions, or that it provides inaccurate or biased results etc. However, the law should not protect illegitimate commercial interests such as the provision of low-quality or harmful services and products. Not even the breach of confidence doctrine will protect information about ‘iniquity’.[152]
Finally, exceptions to the ‘immediate access’ provisions, as listed under s 121(2) of the GIPA Act, will be applied by contractors themselves, not by government agencies, and without the need to balance competing public interests.[153] As discussed above, when an agency is requested to disclose business-related information that they hold, they are required to balance the public interests in favour of and against the disclosure of the information; the former’s prevalence is presumed under law.[154] When the information is held by a government contractor, the contractor will decide whether the requested information falls under any of the categories listed under s 121(2). They are not required to undertake a balancing and weighting of different public interests. In practice, this enables the contractors to easily deny the disclosure of information, as long as such disclosure is likely to cause them any ‘substantial commercial disadvantage ... whether at present or in the future’.[155]
In conclusion, government information held by third parties has much weaker public access and transparency guarantees. The scope of information that third parties are required to disclose (if such contractual duty exists at all) is quite narrow and vague, while the exceptions to these disclosure duties are overly broad and absolute. Keeping in mind the increasing importance of this provision in practice, there is a need for new solutions that would ensure efficient public access to information held by government contractors.
The above analysis has demonstrated the current challenges that trade secrets and other business information cause in ensuring transparency around ADMS in NSW. This Part develops three blocks of proposals on how these challenges could be addressed. These proposals are underpinned by a need to retain
and strengthen government transparency — and the human right to
information — in the context of an ever more digitised and automated government that increasingly outsources the development and deployment of automated government tools.[156] Indeed, if governments adopt proprietary ADMS that impact the structure, operation or decision-making procedures of the government, information about such systems ought to be made public.[157] The involvement of third parties in the automation of government should not diminish government transparency. On the contrary, emerging technologies should enable governments to increase transparency and address the asymmetry of information that is held by the government and by the public, as well as contribute to further democratisation of government institutions.
As Bloch-Wehba summarised:
[W]hen agencies adopt and rely upon proprietary materials or software in making decisions, those materials should generally be considered matters of public record — regardless of the licensing or contractual provisions that govern. Indeed, allowing contracts between the government and its vendors to remove the infrastructure of decision-making from public control reflects a formalistic approach that privileges the private sector’s economic and political power while virtually eviscerating the purposes of the statutory protections.[158]
At the same time, it is unlikely that governments will agree to withdraw the protection of trade secrets and other business information in public-private partnerships entirely.[159] This might lead to undesirable consequences, such as the unwillingness of businesses to collaborate with governments in developing government automation solutions, or lower quality technologies being offered to the government. The below proposals seek to establish a new balance between the public need to access proprietary information related to government ADMS and the protection of third-party interests. While these proposals directly refer to NSW legislation, they would be relevant for other states and territories of Australia when reconsidering the protection of business interests in their respective legislation.
The first proposal is to reduce protection for business-related information, including trade secrets, held by the government. As could be seen from the analysis above, increasing government collaborations with the private sector will result in even more business-related information being held by the government (eg in government contracts). However, strong statutory protections for such information, such as the ‘business interests’ exception under the GIPA Act, impedes the release of such information to the public. In order to facilitate public access to business-related information around ADMS utilised by the NSW government, two actions are recommended: (1) narrowing down the protection for ‘commercial-in-confidence’ information and other business information; and (2) improving the interest balancing test by adding another factor that specifically encourages the disclosure of information about automated government systems.
Narrower protection for both confidential and other business information held by the government is recommended for several reasons. First, the main object of the GIPA Act is ‘to open government information to the public’ with a goal ‘to maintain and advance a system of responsible and representative democratic Government that is open, accountable, fair and effective’.[160] While this goal should be balanced against other legitimate public interests,[161] including the confidentiality interests of third parties, there is no reason why the GIPA Act should grant protection to trade secrets that is more extensive than that granted under a general breach of confidence doctrine.[162]
Second, the Australian breach of confidence doctrine already grants broad protection to trade secrets. In contrast to international law[163] and trade secret laws in many other jurisdictions,[164] Australian breach of confidence law does not explicitly require information to have an independent commercial value, which demonstrates its broader scope of protection. In order to find a breach of confidence, it is sufficient to prove that the information is confidential, was imparted in a way that creates a duty of confidentiality and was used without authorisation.[165] The broad protection offered under the breach of confidence doctrine could thus be sufficient to protect the legitimate interests of government contractors.
Third, courts in other jurisdictions have given narrower protection to trade secrets when dealing with freedom of information requests, as compared to usual trade secret cases. For instance, in case law concerning the US Freedom of Information Act (‘US FOIA’),[166] US courts have interpreted the term ‘trade secret’ to have a more limited meaning than it does under the US FOIA.[167] According to Brauneis and Goodman,
the government may only withhold records under Exemption 4 [of the US FOIA] for ‘a secret, commercially valuable plan, formula, process, or device that is used [in connection with] trade commodities and that can be said to be the end product of either innovation or substantial effort’.[168]
Further, ‘there must be “a direct relationship between the information at issue and the productive process”, rather than merely “collateral business confidentiality”’.[169] This suggests that ‘not all algorithmic processes that a vendor might consider to be trade secrets in the commercial sphere should count as trade secrets for open records exemption purposes’.[170]
Keeping this in mind, there is no rationale to extend protection for commercial confidential information under the GIPA Act beyond the protection granted under the breach of confidence doctrine. Overly extensive protection of business information under the GIPA Act undermines the ultimate goal of the Act and inhibits transparency and accountability of the government.[171] Thus, instead of an overbroad ‘commercial-in-confidence’ definition, at the very least, the GIPA Act could make the protection of commercial confidential information compatible with its protection under the breach of confidence doctrine.
Further, it is questionable whether non-confidential commercial information requires protection at all. As argued above, if a third party provided the government with the information without any confidentiality restrictions, its commercial value and significance is assumingly so limited that its public disclosure does not need to be prohibited. Accordingly, the GIPA Act could envisage that, in order for business information to be protected under the legislation, it should be of commercial significance to the third party (ie provide a competitive advantage) and they should have made efforts to keep it confidential (ie disclosed it in circumstances importing the duty of confidentiality). In comparison, similar exceptions under the US FOIA generally only protect trade secrets and other confidential and privileged information.[172] This additional limitation for the protection of business interests under the GIPA Act would encourage third parties, before disclosing information to the government, to carefully assess which information is commercially valuable and should therefore be kept confidential.
As a second proposal, NSW government agencies could be further encouraged to disclose ADMS-related information through a revised public interest test.[173] Currently, when deciding whether to release proprietary information about ADMS, agencies should weigh the business risks of such disclosure against a list of general factors in favour of such disclosure.[174] Examples of factors in favour of disclosure include information that would: ‘promote open discussion of public affairs, enhance Government accountability or contribute to positive and informed debate on issues of public importance’;[175] ‘inform the public about the operations of agencies and, in particular, their policies and practices for dealing with members of the public’;[176] ‘ensure effective oversight of the expenditure of public funds’;[177] and ‘reveal or substantiate that an agency ... has engaged in misconduct or negligent, improper or unlawful conduct’.[178]
These examples, however, do not specifically refer to the digital or automated government systems that are often developed through public-private partnerships. As explained above, such collaborations frequently result in a lack of public access to information about the systems. At the same time, systems that are based on new and emerging technologies might have elevated risks and often lack established quality assurance and oversight mechanisms.[179] In such cases, the public oversight and quality of such systems should be prioritised over the commercial interests involved. To do so, freedom of information laws could establish mechanisms that would encourage agencies to disclose information about automated government systems.
As one of the possible options for the GIPA Act, an additional factor could be added to the list of factors weighing in favour of the disclosure of government information. This factor could consider the use of automation as a factor favouring the disclosure of information. A similar recommendation was made by the NSW IPC, but in relation to AI technologies only.[180] Such a factor, it is recommended, should refer to all automated systems or processes (not only to AI), as even relatively simple software which does not amount to AI might have significant impacts on individuals and thus pose significant risks.[181] An additional factor in favour of disclosure would give a stronger basis for government agencies to disclose information about ADMS, even if such disclosure affects certain business interests.
Apart from improving disclosure of information held by government, it is also necessary to improve access to information held by government contractors. Keeping in mind that public-private partnerships are likely to increase in the future and that government contractors are likely to hold even more information about government ADMS that they develop or run, it is very important that the public has appropriate access to information held by these third parties. The above analysis has demonstrated that current rules regulating access to such information under the GIPA Act — especially s 121 (the ‘immediate access’ provisions) — are largely inadequate in the modern context of government and private sector partnerships. Three reforms to s 121 are recommended to improve access to information held by NSW government contractors.
The first proposal is to expand s 121 of the GIPA Act so that it not only covers contractors that provide direct services to the public but also any third parties which provide ADMS tools or services that facilitate the provision of government services.[182] This would help address the first problem related to
s 121; namely, its application to a narrow group of government contracts only.
Expanding the scope of s 121 is necessary to reflect the diversity and complexity of government and private sector partnerships in the context of government automation.[183] In many cases, government contractors do not directly provide services to the public on behalf of the agency but ‘merely’ provide algorithmic or automated tools to assist the government with their functions. Government reliance on these tools, however, may have significant effects on individuals’ rights and legitimate interests. As mentioned above, these tools have already automatically issued debt notices[184] and garnishee orders to individuals,[185] played an instrumental role in calculating rental subsidies,[186] and influenced parole decisions.[187] The significance of such tools is not necessarily less — and is sometimes even higher — than the significance of third-party services directly provided to the public. When information about these tools and how they work is held by government contractors, the law should ensure that individuals have the right to access this information. In addition, without expanding the scope of s 121, the asymmetry of the information held by the government and its partners, and information that the public can access, will further increase and is likely to result in unjustified limitations on the human right to information.[188]
Second, the immediate access provisions that are currently listed under s 121(2) of the GIPA Act are general and vague in nature, and thus provide little legal certainty as to the information to which government should have immediate access.[189] These provisions need to be defined in a clear and more specific way to increase legal certainty and predictability for all parties involved. The GIPA Act provisions or the corresponding NSW IPC guidance[190] should more clearly list the types of information about government ADMS to which government contractors are obliged to provide access and under what conditions. For instance, this could be system development and functioning information (eg algorithmic parameters, training and validation information), system external testing, certification and auditing information.[191] The UK Algorithmic Transparency Standard provides useful guidance on what types of information about ADMS could be made publicly available.[192] It could be made clear that while some of this information might be made accessible to the government agency only (ie under confidentiality restrictions), other information needs to be provided for public access purposes (ie free from confidentiality claims).[193] This will ensure that the public has access to the third-party information that is necessary for government transparency purposes.
Thirdly, it is recommended that exceptions to the ‘immediate access’ provisions, as listed under s 121(2) of the GIPA Act, are reconsidered. As demonstrated above, these exceptions are either broad, vague or inadequate, and make it very easy for contractors to reject the request for information as long as certain commercial interests are involved.[194] Current provisions are thus very protective of third-party commercial interests and make it barely possible for the government to fulfil its transparency and accountability duties.
Instead of these vague and broad provisions, more specific exceptions to the ‘immediate access’ provisions could be considered. For instance, such exceptions might focus on trade secrets only.[195] Namely, if the information to which the government wants access constitutes commercial confidential information as defined under general law, this would be a legitimate ground for the third party to reject a disclosure of such information. However, if the requested information is not confidential in nature and the party cannot show how its disclosure would lead to a significant commercial disadvantage, its disclosure should not be pre-empted. In any case, the exceptions for the disclosure of information held by government contractors should be compatible with those applicable to business information held by the government.
The above two solutions have the potential to improve access to third-party information, both held by the government and by government contractors. However, they will not fully solve the problems identified earlier. For instance, even if the above recommendations are adopted, it will remain impossible to publicly disclose information that qualifies as confidential commercial information or trade secrets (either held by the government or by a third party), as disclosure of such information might lead to government liability under the breach of confidence doctrine. If the GIPA Act protects only confidential commercial information, contractors will start using trade secrets even more intensely; for instance, by routinely requesting government agencies to sign non-disclosure agreements.[196] Further, even if laws provided protection only to confidential information that provides a significant commercial advantage, as proposed above, an abundance of publicly relevant information (eg about algorithmic parameters and validation) might prove to be commercially significant and thus excluded from public disclosure duties. For these reasons, additional measures to improve public access to important information about government ADMS need to be considered.
One such possible measure is to ensure that the government leverages its procurement powers to acquire, or using the GIPA Act’s terminology, to get a ‘hold’ of information that is essential for public transparency purposes and without any confidentiality restrictions.[197] In particular, before collaborating with third parties to develop or implement government ADMS, governments should require third parties to commit to the disclosure of information that is essential for government public transparency purposes.[198] This information can be provided on a one-off basis or continuously, and it can be licensed or owned by the government. However, for the purposes of the GIPA Act, it should be considered as ‘held’ by the government agency. Further, government contractors would be required to provide this essential information without any confidentiality restrictions; otherwise, the government might not be able to use this information for public disclosure purposes.[199]
This solution would address several of the problems identified above. First, if such essential information is acquired and held by the government, it could be easily disclosed to the public, either through proactive or reactive transparency pathways. The government would not need to request such information from the contractor, which requires relying on their disclosure procedures and trusting that the third party does not misuse and inappropriately apply the exceptions to their disclosure duties. Further, if the government possesses information without commercial confidentiality restrictions, trade secrets, as a barrier to government transparency, would also be removed. There would be no need for the agency to balance possible conflicting public interests (eg an interest not to disclose trade secrets) and the agency would not be unable to disclose trade secrets due to legal liability risks. Other commentators agree that despite public-private partnerships, the government should find solutions that enable them to provide unrestricted access to the information relevant for transparency purposes.[200]
The government should be able to get hold of such essential information during procurement procedures.[201] As Bloch-Wehba suggests, ‘the transparency problem raised by algorithmic governance as it is presented today results largely from procurement practices that fail to foreground the public interest’.[202] During negotiations, the government could require the contractors to provide such information without confidentiality restrictions.[203] If a contractor does not agree to commit to the unrestricted disclosure of essential information, the government should not enter into contracts with such a party that is not ready to support government transparency obligations.[204]
As suggested by Brauneis and Goodman, this is possible in practice and some government institutions in the US have already been successfully negotiating contracts with much narrower non-disclosure duties or negotiating ownership over confidential information.[205] In the case of commercial contractors (as opposed to not-for-profit organisations providing tools for free), governments arguably have ‘additional leverage’ in negotiations as they are paying for the services.[206]
The implementation of new contracting practices might require a revision of procurement laws and practices as well as continuous monitoring of their implementation.[207] For example, a previously proposed Act of the Washington state legislature required ADMS to have several transparency and accountability-enhancing features, including that they be open to audit and inspection by state agencies.[208] It also stipulated that procurement contracts for ADMS contain non-disclosure provisions or other obstacles to transparency.[209] This proposed Act is an example of how procurement policies around government ADMS could be improved.[210]
In order to implement the above proposal, the remaining challenging task is to determine what constitutes ‘essential’ information — ie what information is so ‘essential’ for public transparency purposes that the government should require third parties to disclose it despite possible associated commercial interests.
Australian and international government institutions and public organisations have made preliminary attempts to list the types of information about ADMS that should be made publicly available, but there is no consensus on this question yet. For instance, the NSW IPC has suggested that agencies should identify ‘datasets and other forms of digital information and inventories of machine enhanced decision-making in Agency Information Guides’, and ensure that ‘documents/information that supports digital government and automated decision-making ... are available as open access information’.[211] The NSW IPC has also recommended that agencies ‘[e]nsure that open access [information] includes a statement of use, inputs, and a description of the operation of the AI system’.[212] The NSW Ombudsman has suggested that
agencies should consider releasing draft specifications, code and even ‘beta’ versions of new machines to draw on external expertise and help to identify flaws or potential improvements before the technology is put into operation.[213]
Under the UK Algorithmic Transparency Recording Standard, government agencies are required to provide information about ownership and responsibility, description and rationale, decision-making processes, technical specifications and data, as well as risk mitigations and impact assessments.[214] Under the EU’s proposal for AI regulations, information to be included in the register of high-risk technologies includes a general description of the technology, a detailed description of its elements and development, detailed information about the monitoring, functioning and control of the AI system, and risk management and post-market evaluation systems.[215] Especially detailed transparency provisions are available in the World Economic Forum white paper on facial recognition technologies in law enforcement.[216] Civil organisations and academic commentators have also developed various proposals on what effective or meaningful government transparency around ADMS would entail.[217]
While a comprehensive public discussion and a separate study are required to determine the types of information that are ‘essential’ for public
transparency purposes, two main perspectives — a ‘stakeholder’ and a ‘system’
perspective — in selecting such essential information are proposed below.
Firstly, according to a ‘stakeholder’ perspective, governments should keep in mind that different stakeholder groups require different levels of transparency about government automation and should ensure that each stakeholder group has access to sufficient information to meet their needs.[218] In particular, governments should meet the transparency needs of non-expert members of the public that have limited knowledge about government ADMS and require relatively general and user-friendly information. Governments should also aim to meet the transparency needs of members of the public such as Non-Government Organisations and researchers, who might have expert knowledge about the systems or their regulation, and require more in-depth and detailed technical information or other quality assurance information.[219] In this latter case, the government would have to balance public interests against the interests of third parties, as it might not be reasonable or justified to require government contractors to disclose all of the technical information that experts may seek. Despite this, governments should ensure that at least some of the relevant information sought by experts is made available to make certain that they can effectively perform a public oversight function. For instance, even if exhaustive information about algorithmic parameters, training and validation cannot be disclosed due to trade secrets, information such as external validation, auditor’s reports and independent legal evaluation reports should be publicly available.
Second, a ‘system’ perspective suggests that ADMS should be subject to different transparency standards depending on the type of system or, more specifically, depending on the level of impact or risk that it poses to the rights and legitimate interests of individuals. If the technology has minimal effects on human rights or legitimate interests (eg automated traffic light systems), minimal information will need to be provided. If an ADMS has medium or significant/high effects on individual rights or interests (eg if it influences a decision affecting an individual’s legal rights or social and economic interests), a high level of transparency about the technical and legal quality of the system should be ensured. The risk or impact of AI is a criterion that has been suggested for regulating AI both in Australia[220] and overseas.[221]
In summary, as governments are increasingly using sophisticated algorithms in their decision-making, which are likely to negatively affect the legitimate interests of their citizens, it becomes indispensable to maintain and increase public transparency around such algorithms. Focusing on the case study of NSW, this article has demonstrated that current transparency around government ADMS in NSW is insufficient, with transparency laws such as the GIPA Act containing significant gaps in ensuring transparency in the age of government automation. This article has proposed a range of legal reforms and improved procurement practices which are expected to assist in enhancing transparency around government automated decision-making as well as ensure better government accountability and improved public service delivery.
* PhD, LLM (Munich), Senior Lecturer at Macquarie University Law School. This article is the outcome of a research project, R[1]NO to BlackBox: Towards More Transparent and Explainable AI in Government’, funded under the Macquarie University Outside Studies Program. My sincere thanks to Serena Abrahams for her editorial assistance.
[1] Digital Transformation Agency (Cth), Digital Government Strategy: Accelerating the Digital Future of Our Australian Public Service (Report, 2021) 3.
[2] Australian Government, Budget 2021–22: Budget Strategy and Outlook (Budget Paper No 1, 11 May 2021) 2.
[3] ‘Beyond Digital’, Digital NSW (Web Page) <https://www.digital.nsw.gov.au/strategy>, archived at <https://perma.cc/9JT6-9DW7>.
[4] ‘Artificial Intelligence Strategy’, Digital NSW (Web Page) <https://www.digital.nsw.gov.au/policy/artificial-intelligence/artificial-intelligence-strategy>, archived at <https://perma.cc/4HEV-JCBX>.
[5] See ‘My Health Record’, Office of the Australian Information Commissioner (Web Page) <https://www.oaic.gov.au/privacy/privacy-legislation/related-legislation/my-health-record>, archived at <https://perma.cc/MM6C-6JSU>.
[6] ‘SmartGates’, Australian Border Force (Web Page, 23 September 2019) <https://www.abf.gov.au/entering-and-leaving-australia/smartgates/overview>, archived at <https://perma.cc/D5KM-G2EJ>.
[7] Commonwealth Ombudsman, Centrelink’s Automated Debt Raising and Recovery System: A Report about the Department of Human Services’ Online Compliance Intervention System for Debt Raising and Recovery (Report, April 2017) 1 (‘Centrelink’s Automated Debt Raising and Recovery System’).
[8] Transport for NSW, NSW Automated Enforcement Strategy for Road Safety (Report, November 2022) 4.
[9] Revenue NSW, ‘Using Artificial Intelligence To Identify and Support Customers Facing Hardship’, Digital NSW (Web Page, 21 September 2021) <https://www.digital.nsw.gov.au/article/using-artificial-intelligence-to-identify-and-support-customers-facing-hardship>, archived at <https://perma.cc/WVZ3-776C>.
[10] ‘New Algorithm Tool Designed To Identify Sepsis Risk in ED’, eHealth NSW (Web Page, 9 September 2022) <https://www.ehealth.nsw.gov.au/news/algorithm-tool-to-identify-sepsis>, archived at <https://perma.cc/K2GC-NAFV>.
[11] Cary Coglianese and David Lehr, ‘Regulating by Robot: Administrative Decision Making in the Machine-Learning Era’ (2017) 105(5) Georgetown Law Journal 1147, 1152–3.
[12] See, eg, Carolin Kemper, ‘Kafkaesque AI? Legal Decision-Making in the Era of Machine Learning’ (2020) 24(2) University of San Francisco Intellectual Property and Technology Law Journal 251, 252–3; Christine Kumar, ‘The Automated Tipster: How Implicit Bias Turns Suspicion Algorithms into BBQ Beckys’ (2020) 72(1) Federal Communications Law Journal 97, 99–100; Anna Huggins, ‘Addressing Disconnection: Automated Decision-Making, Administrative Law and Regulatory Reform’ [2021] UNSWLawJl 37; (2021) 44(3) University of New South Wales Law Journal 1048, 1076;
Yee-Fui Ng, ‘Institutional Adaptation and the Administrative State’ [2021] MelbULawRw 4; (2021) 44(3) Melbourne University Law Review 889, 915–24; Yee-Fui Ng and Maria O’Sullivan, ‘Deliberation and Automation: When Is a Decision a “Decision”?’ (2019) 26(1) Australian Journal of Administrative Law 21, 21–3, 33.
[13] NSW Ombudsman, The New Machinery of Government: Using Machine Technology in Administrative Decision-Making (Report, 29 November 2021) 7 (‘The New Machinery of Government’).
[14] [2022] NSWCATAD 100 (‘O’Brien’).
[15] [2022] NSWCATAD 366 (‘Ireland’).
[16] O’Brien (n 14) [2]–[5] (Senior Member McAteer); ibid [1]–[7] (Senior Member Ransome). For further discussion of these cases, see below at nn 66–74 and accompanying text.
[17] Various reasons for transparency are well summarised in Michele Loi, Automated Decision-Making Systems in the Public Sector: An Impact Assessment Tool for Public Authorities (Report, June 2021) 18–21; Lyria Bennett Moses and Louis de Koker, ‘Open Secrets: Balancing Operational Secrecy and Transparency in the Collection and Use of Data by National Security and Law Enforcement Agencies’ [2017] MelbULawRw 32; (2017) 41(2) Melbourne University Law Review 530, 534–6, 548.
[18] Organisation for Economic Co-Operation and Development, Modernising Government: The Way Forward (2005) 29. For a short overview of the history of open government, see Anita Stuhmcke, Advancing the Objects of the Government Information (Public Access) Act 2009 (NSW): An International Comparative Evaluation of Measures Used To Promote Government Information Release (Report, 23 June 2015) 13–16; Bernd W Wirtz and Steven Birkmeyer, ‘Open Government: Origin, Development, and Conceptual Perspectives’ (2015) 38(5) International Journal of Public Administration 381, 382–4; Fons Wijnhoven, Michel Ehrenhard and Johannes Kuhn, ‘Open Government Objectives and Participation Motivations’ (2015) 32(1) Government Information Quarterly 30, 31–2.
[19] See, eg, International Covenant on Civil and Political Rights, opened for signature 19 December 1966, 999 UNTS 171 (entered into force 23 March 1976) art 19(2).
[20] For analyses of various challenges related to emerging government technologies, see Han-Wei Liu, Ching-Fu Lin and Yu-Jie Chen, ‘Beyond State v Loomis: Artificial Intelligence, Government Algorithmization and Accountability’ (2019) 27(2) International Journal of Law and Information Technology 122, 124–6; Lyria Bennett Moses and Edward Santow, ‘Accountability in the Age of Artificial Intelligence: A Right to Reasons’ (2020) 94(11) Australian Law Journal 829, 829–30; Emily Berman, ‘A Government of Laws and Not of Machines’ (2018) 98(1) Boston University Law Review 1277, 1326–31; Noah Bunnell, ‘Remedying Public-Sector Algorithmic Harms: The Case for Local and State Regulation via Independent Agency’ (2021) 54(2) Columbia Journal of Law and Social Problems 261,
265–8.
[21] See also Robert Brauneis and Ellen P Goodman, ‘Algorithmic Transparency for the Smart City’ (2018) 20 Yale Journal of Law and Technology 103, 108–9.
[22] See, eg, Erik Bakke, ‘Predictive Policing: The Argument for Public Transparency’ (2018) 74(1) New York University Annual Survey of American Law 131, 133–41.
[23] For example, in a 2020 Australian Human Rights Commission survey, ‘85% of survey participants indicated it was very important, or somewhat important, to be informed that a computer program has made an automated decision affecting them’: Australian Human Rights Commission, Human Rights and Technology (Final Report, 1 March 2021) 61.
[24] Government transparency is the main goal of federal and state freedom of information acts: see, eg, Freedom of Information Act 1982 (Cth) s 3 (‘FOI Act (Cth)’); Archives Act 1983 (Cth)
s 2A; Public Interest Disclosure Act 2013 (Cth) s 6; Government Information (Public Access) Act 2009 (NSW) s 3 (‘GIPA Act’); Freedom of Information Act 1982 (Vic) s 3. Since 2015, the Australian government has also been a member of the Open Government Partnership: ‘Australia’, Open Government Partnership (Web Page) <https://www.opengovpartnership.org/members/australia>, archived at <https://perma.cc/6V7X-5V25>.
[25] See, eg, Department of Industry, Science and Resources (Cth), ‘Australia’s AI Ethics Principles’, Australia’s Artificial Intelligence Ethics Framework (Web Page, 7 November 2019) <https://www.industry.gov.au/publications/australias-artificial-intelligence-ethics-framework/australias-ai-ethics-principles>, archived at <https://perma.cc/ZD7J-CU6T>. According to Digital Transformation Agency (Cth) (n 1), the Agency ‘will be open and transparent in [its] interactions by clearly communicating [its] processes including the way [it] use[s] data and make[s] decisions’: at 12. See also NSW Government, ‘Artificial Intelligence Ethical Policy’, NSW Digital (Web Page) <https://www.digital.nsw.gov.au/policy/artificial-intelligence/artificial-intelligence-ethics-policy>, archived at <https://perma.cc/CSB6-A7KF>. According to the NSW government, AI must be a ‘solution that is open and transparent so that NSW citizens have access to efficient review mechanisms’.
[26] NSW Ombudsman, The New Machinery of Government (n 13) 7. See also below Part II(B).
[27] While in different contexts these terms might have slightly different meanings, for the purpose of this article, they are used as synonyms.
[28] NSW Ombudsman, The New Machinery of Government (n 13) states ‘[a] key transparency issue arises when an agency engages an external provider for machine technology expertise’: at 62. In its submission to the Australian Human Rights Commission, the Information and Privacy Commission (NSW) states that ‘there is scope to strengthen existing information access laws to better facilitate access to AI-informed decision-making, particularly where governments partner with the private sector and NGOs in using these technologies’: Information and Privacy Commission (NSW), Submission to Australian Human Rights Commission, Human Rights and Technology Discussion Paper (24 February 2020) 19 <https://humanrights.gov.au/sites/default/files/2020-07/10_-_information_and_privacy_commission_nsw_1.pdf>, archived at <https://perma.cc/L9JT-SRAE>. See also Jake Goldenfein, ‘Algorithmic Transparency and Decision-Making Accountability: Thoughts for Buying Machine Learning Algorithms’ in Cliff Bertram, Asher Gibson and Adriana Nugent (eds), Closer to the Machine: Technical, Social, and Legal Aspects of AI (Office of the Victorian Information Commissioner, August 2019) 41, 42–3; Rebecca Wexler, ‘Life, Liberty, and Trade Secrets: Intellectual Property in the Criminal Justice System’ (2018) 70(5) Stanford Law Review 1343, 1346, 1349–50.
[29] See below Part II(B).
[30] For an analysis of transparency and ADMS problems under the FOI Act (Cth) (n 24), see
Yee-Fui Ng et al, ‘Revitalising Public Law in a Technological Era: Rights, Transparency and Administrative Justice’ [2020] UNSWLawJl 37; (2020) 43(3) University of New South Wales Law Journal 1041, 1052–6.
[31] For an analysis of state transparency laws in the United States (‘US’), see generally Christina Koningisor, ‘Transparency Deserts’ (2020) 114(6) Northwestern University Law Review 1461.
[32] See, eg, Ayo Næsborg Anderson and Hanne Marie Motzfeldt, ‘Different Aspects of Transparency in Digital Government: The Danish Case’ (Conference Paper, European Conference on Digital Government, 24 October 2019) 81–6. Anderson and Motzfeldt suggest transparency in digital government could be divided into transparency of data, transparency of algorithms, transparency of administrative decisions and democratic transparency. See also Tom M van Engers and Dennis M de Vries, ‘Governmental Transparency in the Era of Artificial Intelligence’ in Michal Araszkiewicz and Víctor Rodríguez-Doncel (eds), Legal Knowledge and Information Systems (IOS Press, 2019) vol 322, 33, 35–6. See generally van Engers and de Vries (n 32) which discusses the explainability of AI decisions as an element of transparency.
See also Tal Z Zarsky, ‘Transparent Predictions’ [2013] (4) University of Illinois Law Review 1503, 1530–2, which suggests that transparency is not well understood in the algorithmic government decision-making context. For a brief history of government transparency, see Daniel J Metcalfe, ‘The History of Government Transparency’ in Padideh Ala’i and Robert G Vaughn (eds), Research Handbook on Transparency (Edward Elgar Publishing, 2014) 247.
[33] Department of Industry, Science and Resources (Cth), ‘Australia’s AI Ethics Principles’
(n 25).
[34] NSW Government, ‘Mandatory Ethical Principles for the Use of AI’, NSW Digital (Web Page) <https://www.digital.nsw.gov.au/policy/artificial-intelligence/artificial-intelligence-ethics-policy/mandatory-ethical-principles>, archived at <https://perma.cc/JU3D-P7PQ>.
[35] Ibid. This compares with the definition provided in Australian Human Rights Commission, Human Rights and Technology (n 23) 60. According to the Australian Human Rights Commission, transparency requires ‘that any affected individual is notified where artificial intelligence is materially used in making an administrative decision’.
[36] See generally David E Pozen, ‘Transparency’s Ideological Drift’ (2018) 128(1) Yale Law Journal 100, for a discussion on transparency’s historical drift from being a progressive ideal to a libertarian one.
[37] Department of Industry, Science and Resources (Cth), ‘Australia’s AI Ethics Principles’ (n 25).
[38] Cf NSW Ombudsman, The New Machinery of Government (n 13) which proposes strengthening government transparency towards courts, tribunals and disciplinary or regulatory bodies: at 62.
[39] Note that transparency can serve various goals: see, eg, Bakke (n 22) 144. Bakke lists five reasons for transparency: ‘(1) democratic legitimacy; (2) the prevention of misconduct; (3) a remedy for injustices; (4) better substantive policies; and (5) community trust’. Tom van Nuenen et al, ‘Transparency for Whom? Assessing Discriminatory Artificial Intelligence’ (2020) 53(11) Computer 36, 37, 42–3, argues that transparency should be understood by looking at the issue from the perspectives of various stakeholders. See also Zarsky (n 32)
1530–50.
[40] See, eg, Frank Bannister and Regina Connolly, ‘The Trouble with Transparency: A Critical Review of Openness in E-Government’ (2011) 3(1) Policy and Internet 1, 6–7, 9–18; Harlan Yu and David G Robinson, ‘The New Ambiguity of “Open Government”’ [2011] 59 UCLA Law Review Discourse 178, 182; van Nuenen et al (n 39) 37; Danielle Moon and Carolyn Adams, ‘Too Much of a Good Thing? Balancing Transparency and Government Effectiveness in FOI Public Interest Decision Making’ (2015) 82 (November) Australian Institute of Administrative Law Forum 28, 29–30.
[41] See, eg, Mark Fenster, ‘The Opacity of Transparency’ (2006) 91(3) Iowa Law Review 885, 893, 902–10; van Nuenen et al (n 39) 37.
[43] Ibid s 6(1).
[44] The list of ‘public interests considerations’ can be found under s 14 of the GIPA Act (n 24).
[45] Information and Privacy Commission (NSW), Automated Decision-Making, Digital Government and Preserving Information Access Rights: For Agencies (Fact Sheet, August 2022) 2 (‘NSW IPC Fact Sheet’).
[47] Ibid s 9(1).
[48] ‘Open Government, Open Data & Public Participation’, Information and Privacy Commission (NSW) (Web Page) <https://www.ipc.nsw.gov.au/information-access/open-government-open-data>, archived at <https://perma.cc/5X7G-HMF4>; Information and Privacy Commission (NSW), Charter for Public Participation: A Guide To Assist Agencies and Promote Citizen Engagement (Guide, June 2018) 4.
[50] ‘Open Government, Open Data & Public Participation’ (n 48).
[51] Ibid.
[52] See above n 34 and accompanying text.
[53] See, eg, Information and Privacy Commission (NSW), 10 Year GIPA Data Analysis Overview 2010 to 2020 (Report, September 2021) 6 <https://www.ipc.nsw.gov.au/nsw-information-commissioners-10-year-gipa-data-analysis-overview-2010-2020>, archived at <https://perma.cc/G79C-ZU3E> (‘GIPA Data Analysis’). During 2010–20, applications to access government information increased from 6,000 to 13,690.
[54] Ibid, citing Information and Privacy Commission (NSW), NSW Community Attitudes to Information Access (Web Document, April 2020) <www.ipc.nsw.gov.au/media/3115>.
[55] GIPA Act (n 24) s 6(1); Information and Privacy Commission (NSW), NSW IPC Fact Sheet
(n 45) 1–2.
[56] NSW Ombudsman, The New Machinery of Government (n 13) 7. At the time of writing, the NSW Ombudsman, in collaboration with ADM+S, was conducting a study to map the uses of ADMS in NSW agencies: Loren Dela Cruz, ‘ADM+S Partners with New South Wales Ombudsman To Analyse the Use of Automated Systems in NSW Government Sectors’, ADM+S (Web Page, 23 January 2023) <https://www.admscentre.org.au/adms-partners-with-the-new-south-wales-ombudsman-to-analyse-the-use-of-automated-systems-in-nsw-government-sectors/>, archived at <https://perma.cc/2N6Q-L425>.
[57] Information and Privacy Commission (NSW), GIPA Data Analysis (n 53) 4.
[58] NSW Ombudsman, The New Machinery of Government (n 13) 2–3; Lucy Cormack, ‘Revenue NSW’s Automated System Took Fines from People’s Bank Accounts’, The Sydney Morning Herald (online, 30 November 2021) <https://www.smh.com.au/politics/nsw/revenue-nsw-s-automated-system-took-fines-from-people-s-bank-accounts-20211129-p59d2m.html>, archived at <https://perma.cc/LG59-78G2>.
[60] ‘Department of Customer Service Agency Information Guide’, NSW Government (June 2024, Web Page) <https://www.nsw.gov.au/departments-and-agencies/customer-service/access-to-information/department-of-customer-service-agency-information-guide>, archived at <https://perma.cc/6DHY-QPL8>. Some brief information about this second tool was announced on the Digital NSW web page as an example of a positive use of AI: ibid.
[61] Information and Privacy Commission (NSW), NSW IPC Fact Sheet (n 45) 2.
[62] Information and Privacy Commission (NSW), GIPA Data Analysis (n 53) 6.
[63] ‘5b: Did Applicants Get What They Asked for?’, Information and Privacy Commission (NSW) (Web Page) <https://www.ipc.nsw.gov.au/information-access/agency-gipa-dashboard/gipa-dashboard>. The latter number includes full refusal of information (6%), information already publicly available (2%) and applications withdrawn (6%).
[64] See generally ‘Agency GIPA Dashboard’, Information and Privacy Commission (NSW) (Web Page) <https://www.ipc.nsw.gov.au/information-access/gipa-dashboard>, archived at <https://perma.cc/4MYD-7DTD>.
[65] The release rates for general information and for information related to digital government have both been around 65%: Information and Privacy Commission (NSW), GIPA Data Analysis (n 53) 6, 8.
[66] O’Brien (n 14) [2]–[3], [8] (Senior Member McAteer).
[67] Ibid [10], [40].
[68] Ibid [13].
[69] Ibid [99], [117]–[118].
[70] Ibid [96], [100].
[71] Ibid [102]–[103]. For a useful summary of the decision, see ‘Case Note: O’Brien v Secretary, Department Communities and Justice [2022] NSWCATAD 100’, Information and Privacy Commission (NSW) (Web Page) <https://www.ipc.nsw.gov.au/case-note-obrien-v-secretary-department-communities-and-justice-2022-nswcatad-100>, archived at <https://perma.cc/DL46-4K76>.
[72] Ireland (n 15) [8]–[9], [40]–[41] (Senior Member Ransome).
[73] Ibid [4]–[5], [20].
[74] Ibid [45]–[50].
[75] See above Part II.
[76] For a broader discussion on how technology use in government affects citizens’ views on government transparency, see David Valle-Cruz, Rodrigo Sandoval-Almazan and J Ramon Gil-Garcia, ‘Citizens’ Perceptions of the Impact of Information Technology Use on Transparency, Efficiency and Corruption in Local Governments’ (2016) 21(3) Information Polity 321, 321–4, 330–1.
[77] Brauneis and Goodman (n 21) 152. See also Ng et al (n 30) which states: ‘[T]he FOI Act does not contain any specific requirements concerning the creation or retention of data. There is no general statutory requirement to create documents and whether or not they are retained is determined by the requirements in the Archives Act 1983 (Cth)’: at 1054.
[78] See, eg, O’Brien (n 14) [99] (Senior Member McAteer).
[79] For example, in Re Schouten and Secretary, Department of Education, Employment and Workplace Relations [2011] AATA 365, the departmental representative at the Administrative Appeals Tribunal hearing was unable to explain how the applicant’s rate of youth allowance was determined because of the complexity of the database and its programming: at [38]–[39] (Senior Member Britton).
[80] See, eg, GIPA Act (n 24) ss 13–15.
[81] See, eg, Sharon K Sandeen and Tanya Aplin, ‘Trade Secrecy, Factual Secrecy and the Hype Surrounding AI’ in Ryan Abbott (ed), Research Handbook on Intellectual Property and Artificial Intelligence (Edward Elgar Publishing, 2022) 443, 443–4; Ana Nordberg, ‘Trade Secrets, Big Data and Artificial Intelligence Innovation: A Legal Oxymoron?’ in Jens Schovsbo, Timo Minssen and Thomas Riis (eds), The Harmonization and Protection of Trade Secrets in the EU: An Appraisal of the EU Directive (Edward Elgar Publishing, 2020) 192, 192–203; Sonia K Katyal, ‘The Paradox of Source Code Secrecy’ (2019) 104(5) Cornell Law Review 1183, 1186, 1225–50; Wexler (n 28) 1346–56.
[82] Information and Privacy Commission (NSW), Report on the Operation of the Government Information (Public Access) Act 2009: 2020–2021 (Report, 2022) 5.
[83] Information and Privacy Commission (NSW), Digital Restart Fund: Assessing Information Access and Privacy Impacts (Report, September 2022) 8 (‘NSW Digital Restart Fund’).
[84] Ibid.
[85] See ‘NSW Contracts Register’, Infrastructure NSW (Web Page) <https://www.infrastructure.nsw.gov.au/accessing-information/nsw-contracts-register/>, archived at <https://perma.cc/TWR8-YS89>.
[86] Sandin and Aplin (n 81) 443–5; Nordberg (n 81) 199–202, 210–17; Katyal (n 81) 1225–32. See also Bunnell (n 20) 266, 276–8; Bakke (n 22) 156–63.
[87] See, eg, Hannah Bloch-Wehba, ‘Access to Algorithms’ (2020) 88(4) Fordham Law Review 1265, 1290–5, which discusses how government service outsourcing and third-party involvement in automating government services causes various challenges. See also Gillian E Metzger, ‘Private Delegations, Due Process, and the Duty To Supervise’ in Jody Freeman and Martha Minow (eds), Government by Contract: Outsourcing and American Democracy (Harvard University Press, 2009) 291, 299–300. Metzger observes that ‘with expanding privatization, private entities increasingly are undertaking adjudications on behalf of the government’: at 299. See also
Ng (n 12) 907–15.
[88] See above Part II(B).
[89] The NSW Digital Restart Fund is an excellent example of this: Information and Privacy Commission (NSW), NSW Digital Restart Fund (n 83).
[90] See above n 87. See also Deirdre K Mulligan and Kenneth A Bamberger, ‘Procurement as Policy: Administrative Process for Machine Learning’ (2019) 34(3) Berkeley Technology Law Journal 781, 785–8.
[91] See, eg, Goldenfein (n 28) 42–3, 46; Wexler (n 28) 1348–53; Bakke (n 22) 158–63.
[92] Rita Matulionyte, ‘Reconciling Trade Secrets and Explainable AI: Face Recognition Technology as a Case Study’ (2022) 44(1) European Intellectual Property Review 36, 36 (‘Reconciling Trade Secrets’); Katarina Foss-Solbrekk, ‘Three Routes to Protecting AI Systems and Their Algorithms under IP Law: The Good, the Bad and the Ugly’ (2021) 16(3) Journal of Intellectual Property Law and Practice 247, 248, 250–6; Nari Lee, ‘Protection for Artificial Intelligence in Personalised Medicine: The Patent/Trade Secret Tradeoff’ in Jens Schovsbo, Timo Minssen and Thomas Riis (eds), The Harmonization and Protection of Trade Secrets in the EU: An Appraisal of the EU Directive (Edward Elgar Publishing, 2020) 267, 277–8; Bakke (n 22) 162–3.
[93] Matulionyte, ‘Reconciling Trade Secrets’ (n 92) 36.
[94] GIPA Act (n 24) s 14 item 4.
[95] Ibid s 121(2).
[96] Ibid s 4(1) (definition of ‘government information’).
[97] Ibid ss 5–9.
[98] These categories are: (1) ‘Responsible and effective government’; (2) ‘Law enforcement and security’; (3) ‘Individual rights, judicial processes and natural justice’; (4) ‘Business interests of agencies and other persons’; (5) ‘Environment, culture, economy and general matters’;
(6) ‘Secrecy provisions’; and (7) ‘Exempt documents under interstate Freedom of Information legislation’: ibid s 14. NSW IPC refers to each category as an ‘overriding public interest against disclosure’ (‘OPIAD’): see, eg, Information and Privacy Commission (NSW), GIPA Data Analysis (n 53) 7.
[99] GIPA Act (n 24) s 14 item 4.
[100] Ibid s 14 item 4(a).
[101] Ibid s 14 item 4(b).
[102] Ibid s 14 item 4(c).
[103] Ibid s 14 item 4(d).
[104] Notably, when information concerns third-party interests, other categories of exceptions can be invoked too: see, eg, ibid s 14 items 1(d), (g); Male v Kempsey Shire Council [2022] NSWCATAD 39, [318] (Senior Member Starke) (‘Male’).
[105] GIPA Act (n 24) ss 54(1)–(2). See also Information and Privacy Commission (NSW), Guideline 5: Consultation on Public Interest Considerations under Section 54 and Section 54A of the Government Information (Public Access) Act 2009 (NSW) (Report, April 2019) (‘NSW IPC Guideline 5’).
[107] Examples of considerations that could be taken into account are listed under s 12 of the GIPA Act (n 24).
[108] Ibid s 5.
[109] While the GIPA Act (n 24) does not refer to ‘trade secrets’ at all, the ‘commercial-in-confidence’ term comes closest to the trade secret concept and, for the purpose of this article, I use them as synonyms.
[110] For criticisms of the trade secret exception under s 47 of the FOI Act (Cth) (n 24), see Ng et al (n 30) 1055.
[111] GIPA Act (n 24) sch 4 item 1 (definition of ‘commercial-in-confidence provisions’).
[112] See generally MW Bryan et al, A Sourcebook on Equity & Trusts in Australia (Cambridge University Press, 2nd ed, 2020) 308–13.
[113] Coco v AN Clark (Engineers) Ltd (1969) 1A IPR 587, 590 (Megarry J) (‘Coco’). See also Smith Kline & French Laboratories (Australia) Ltd v Secretary, Department of Community Services and Health [1989] FCA 384; (1990) 17 IPR 545, 567–9 (Gummow J); Andrew Stewart et al, Intellectual Property in Australia (LexisNexis Butterworths, 6th ed, 2018) 92–5 [4.5]–[4.6] (citations omitted).
[114] Australian patents and trademarks are published by IP Australia: ‘Our Agency’, IP Australia (Cth) (Web Page) <https://www.ipaustralia.gov.au/about-us/our-agency>, archived at <https://perma.cc/6X3L-JVDD>.
[115] IP Australia (Cth), Australian IP (Report, 2024) 84.
[116] O’Brien (n 14) [91]–[92], [102]–[104] (Senior Member McAteer).
[117] For example, in Male (n 104), the Tribunal was satisfied that, in the circumstances of the case, contract price was commercially sensitive information: at [215], [219] (Senior Member Starke).
[118] In the EU, see Directive (EU) 2016/943 of the European Parliament and of the Council of 8 June 2018 on the Protection of Undisclosed Know-How and Business Information (Trade Secrets) against Their Unlawful Acquisition, Use and Disclosure [2016] OJ L 157/1, art 2(1) (‘EU Trade Secrets Directive’). In the US, see the Uniform Trade Secrets Act § 1(4) (definition of ‘trade secrets’) (ULA 1985) (‘UTSA’) which is a uniform law that has been adopted in 49 states:
‘Trade Secrets Act’, Uniform Law Commission (Web Page, 12 July 2024) <https://www.uniformlaws.org/committees/community-home?CommunityKey=3a2538fb-e030-4e2d-a9e2-90373dc05792>, archived at <https://perma.cc/MS7K-B4TX>. See also Sandeen and Aplin (n 81) 452–9.
[119] Sandeen and Aplin (n 81) 456.
[120] Ibid. See also Camilla A Hrdy, ‘The Value in Secrecy’ (2022) 91(2) Fordham Law Review 557, 559, 565–74.
[121] See Sandeen and Aplin (n 81) 456. See also Brauneis and Goodman (n 21) 159, which states:
Even assuming that the source code and certain details of the model would qualify as trade secrets or confidential information, we sought training materials, existing and planned validation studies, and other documentation concerning the objectives and design choices reflected in the algorithm. It is hard to imagine that most, if any, of this material would qualify for the [trade secret] exemption.
[122] GIPA Act (n 24) s 14 item 4(c).
[123] Ibid s 14 item 4(d).
[124] For a brief explanation as to what constitutes confidential and non-confidential information, see Information and Privacy Commission (NSW), NSW IPC Guideline 5 (n 105) 7 [1.19].
[125] See Saltman Engineering Co Ltd v Campbell Engineering Co Ltd (1948) 65 RPC 203, 215
(Lord Greene MR, Somervell LJ agreeing at 216, Cohen J agreeing at 219); O’Brien v Komesaroff [1982] HCA 33; (1982) 150 CLR 310, 326–7 (Mason J, Aickin J agreeing at 329, Wilson J agreeing at 329, Brennan J agreeing at 329); Johns v Australian Securities Commission [1993] HCA 56; (1993) 178 CLR 408, 432 (Brennan J), 438 (Dawson J), 461 (Gaudron J), 475 (McHugh J).
[126] See also Brauneis and Goodman (n 21) 154, which states that ‘[o]verbroad assertions of confidentiality in response to open records requests are common in the field’.
[127] Stewart et al (n 114) 106–12 [4.17]–[4.22]; Tanya Aplin et al, Gurry on Breach of Confidence: The Protection of Confidential Information (Oxford University Press, 2012) 3 [1.01].
[128] See also Brauneis and Goodman (n 21) 155, referring to the ‘[r]isk averse’ nature of government organisations in such scenarios.
[129] O’Brien (n 14) [91]–[92] (Senior Member McAteer).
[130] Ibid [103]–[104]. This is similar to the application of the trade secret exception in the US where ‘[t]here exists a substantial foundation in both federal and state court decisions for denying open records laws requests for information related to the IP of private government contractors’: Bakke (n 22) 160.
[131] GIPA Act (n 24) s 121(1)(a).
[132] Ibid s 121(1)(b).
[133] Ibid s 121(1)(c).
[134] Ibid s 121(2)(a).
[135] Ibid s 121(2)(b).
[136] Ibid s 121(2)(c).
[137] See, eg, Information and Privacy Commission (NSW), Guide to Section 121 of the GIPA Act for Private Sector Contractors (Fact Sheet, April 2020) 1–2 (‘Section 121 Guide for Private Sector Contractors’).
[138] See, eg, Information and Privacy Commission (NSW), Digital Projects (Fact Sheet, October 2020).
[139] O’Brien (n 14) [64] (Senior Member McAteer).
[140] The NSW Information and Privacy Commissioner submitted:
In this case the service provided by the government agency is private rental subsidies. NPS’s algorithm provides an input to the decision-making process regarding the calculation which is intrinsic to the provision of private rental subsidies. The Tribunal may therefore consider that NPS is providing a service directly to the public on behalf of the agency. Such an interpretation would accord with the objects of the GIPA Act.
Ibid [82] (Senior Member McAteer).
[141] Ibid [89].
[142] Marie Bashir, Governor (NSW), ‘Commencement Proclamation under the Government Information (Public Access) Act 2009’ in New South Wales, Government Gazette of the State of New South Wales, No 84, 25 June 2010, 2901, 2901.
[143] O’Brien (n 14) [63], [90] (Senior Member McAteer).
[144] Ibid [114].
[145] NSW IPC offers template contractual provisions on access to information clauses for government contractors which replicate the problematic wording of s 121: see Information and Privacy Commission (NSW), Guide to Section 121 of the GIPA Act for Agencies (Fact Sheet, April 2020) 2 (‘Section 121 Guide for Agencies’).
[146] GIPA Act (n 24) s 121(1)(a).
[147] Ibid s 121(2)(c). It is noted that exceptions to the ‘immediate access’ rule are to some extent similar, but not identical, to the ‘business interests’ exception that applies to information held by government: see at s 14 item 4. See also above Part III(B).
[148] See above Part III(B)(1).
[149] O’Brien (n 14) [103] (Senior Member McAteer).
[150] Ibid [112]. See also at [103]–[104].
[151] See Copyright Act 1968 (Cth) ss 31(1), 36(1).
[152] In Gartside v Outram (1856) 26 LJ Ch 113, 114, Wood V-C states:
The true doctrine is, that there is no confidence as to the disclosure of iniquity. You cannot make me the confidant of a crime or a fraud, and be entitled to close up my lips upon any secret which you have the audacity to disclose to me relating to any fraudulent intention on your part: such a confidence cannot exist.
[153] Sections 12–14 of the GIPA Act (n 24) apply to government agencies only: see at s 4(1) (definitions of ‘agency’ and ‘government information’).
[154] Ibid s 5.
[155] Ibid s 121(2)(c).
[156] See above Part II.
[157] See, eg, Bloch-Wehba (n 87) 1299–300.
[158] Ibid 1299.
[159] Brauneis and Goodman (n 21) 163.
[161] See ibid s 3(1)(c).
[162] See also Brauneis and Goodman (n 21) 158–9.
[163] See, eg, Marrakesh Agreement Establishing the World Trade Organization, opened for signature 15 April 1994, 1867 UNTS 3 (entered into force 1 January 1995) annex 1C (‘Agreement on Trade-Related Aspects of Intellectual Property Rights’) art 39(2)(b).
[164] See, eg, EU Trade Secrets Directive (n 118) art 2(1)(b); UTSA (n 118) § 1(4)(i).
[165] Stewart et al (n 113) 92 [4.5], quoting Coco (n 113) 590 (Megarry J).
[166] Freedom of Information Act, 5 USC § 552(b)(4) (2018) (‘US FOIA’).
[167] Brauneis and Goodman (n 21) 157.
[168] Ibid, quoting Public Citizen Health Research Group v Food and Drug Administration, [1983] USCADC 159; 704 F 2d 1280, 1288 [3] (Edwards J) (DC Cir, 1983) (‘Public Citizen’).
[169] Public Citizen (n 168) 1287 [1], 1288 [3] (Edwards J), quoting Mark Q Connelly, ‘Secrets and Smokescreens: A Legal and Economic Analysis of Government Disclosures of Business Data’ [1981] (2) Wisconsin Law Review 207, 230. See also Brauneis and Goodman (n 21) 157.
[170] Brauneis and Goodman (n 21) 157.
[171] Cf ibid 154, which states that ‘the agencies have agency’ and ‘could have made more records disclosable simply by reducing the scope of confidentiality’.
[172] There is an exemption for ‘trade secrets and commercial or financial information obtained from a person [that is] privileged or confidential’: US FOIA (n 166) § 552(b)(4). Commercial significance and efforts to keep the information confidential are the core requirements for the protection of trade secrets or confidential information under the US FOIA (n 166): ibid
157–8.
[173] See generally GIPA Act (n 24) s 13.
[174] Ibid ss 12–14.
[175] Ibid s 12(2) note (a).
[176] Ibid s 12(2) note (b).
[177] Ibid s 12(2) note (c).
[178] Ibid s 12(2) note (e).
[179] See above Part II(B).
[180] The NSW IPC published guidance recommending ‘the use of AI as a factor in favour of disclosure of information under the GIPA Act’: Elizabeth Tydd and Samantha Gavel, ‘Scan of the Artificial Intelligence Regulatory Landscape: Information Access & Privacy’, Information and Privacy Commission (NSW) (Web Document, October 2022) <https://www.ipc.nsw.gov.au/sites/default/files/2022-11/IPC_Scan_of_the_Artificial_Intelligence_Regulatory_Landscape_October_2022_0.pdf>.
[181] For example, the Robodebt scheme was not based on complex AI algorithms such as machine learning, but it resulted in immense harm on individuals and on government: Commonwealth Ombudsman, Centrelink’s Automated Debt Raising and Recovery System (n 7) 4.
[182] NSW IPC recommended ‘[e]xpand[ing] information access rights under government contracted services to AI used for decision making’: Tydd and Gavel (n 180).
[183] For a discussion on the variety of private sector partnerships in Europe, see Martijn van den Hurk et al, ‘National Varieties of Public-Private Partnerships (PPPs): A Comparative Analysis of PPP-Supporting Units in 19 European Countries’ (2016) 18(1) Journal of Comparative Policy Analysis 1, 1–3.
[184] One example is the Robodebt case: Commonwealth Ombudsman, Centrelink’s Automated Debt Raising and Recovery System (n 7) 1; Terry Carney, ‘Robo-Debt Illegality: The Seven Veils of Failed Guarantees of the Rule of Law?’ (2019) 44(1) Alternative Law Journal 4, 4–8.
[185] NSW Ombudsman, The New Machinery of Government (n 13) 2–3.
[186] O’Brien (n 14) [3], [8] (Senior Member McAteer).
[187] The risk-assessment tool, ‘COMPAS’, is used by law enforcement in the US ‘to assess the risks of recidivism’: Matulionyte, ‘Reconciling Trade Secrets’ (n 92) 37.
[189] See above Part III(C)(3).
[190] See, eg, Information and Privacy Commission (NSW), Section 121 Guide for Private Sector Contractors (n 137); Information and Privacy Commission (NSW), Section 121 Guide for Agencies (n 145).
[191] According to the NSW IPC, government contracts relating to the provision of AI to government should cover the ‘government/purchaser right to audit/audit logs; notification requirements in circumstances of adverse impacts including complaints or legal action; access to user manuals; training data; [retention of] government data inputs’: Tydd and Gavel (n 180).
[192] Centre Digital and Data Office and Department for Science, Innovation and Technology (UK), ‘Algorithmic Transparency Recording Standard Hub’, Gov.UK (Web Page, 7 March 2024) <https://www.gov.uk/government/collections/algorithmic-transparency-recording-standard-hub>, archived at <https://perma.cc/Q7TT-5Q73>.
[193] See below Part IV(C).
[194] See above Part III(C)(4).
[195] See above Part IV(A)(1) for a similar recommendation.
[196] See Brauneis and Goodman (n 21) 155–6.
[197] Brauneis and Goodman similarly suggest ‘reducing the scope of confidentiality and ensuring government possession of records necessary to explain the algorithms’: ibid 154. See also NSW Ombudsman, The New Machinery of Government (n 13) 62, which states that ‘the tension [between trade secrets and transparency] can be avoided by good procurement practices’.
[198] See also Goldenfein (n 28) 50, 59.
[199] See Bloch-Wehba (n 87) 1308–12. Bloch-Wehba discusses why it is unacceptable to disclose trade secrets under protective orders only, and not to the general public.
[200] See, eg, ibid 1308.
[201] Improved procurement practices as a way to improve government algorithmic accountability is advocated for in Cary Coglianese and Erik Lampmann, ‘Contracting for Algorithmic Accountability’ (2021) 6(3) Administrative Law Review Accord 175, 179; Goldenfein (n 28) 41. The risks associated with the procurement of government ADMS are discussed in Mulligan and Bamberger (n 90) 787–91.
[202] Bloch-Wehba (n 87) 1307.
[203] Ibid 1308.
[204] ‘Informed by the principles of open government, agencies and courts should avoid contracting for proprietary decision-making tools with vendors who require broad secrecy provisions’: ibid 1313.
[205] Brauneis and Goodman (n 21) 164–5.
[206] Ibid 165.
[207] NSW IPC has emphasised on numerous occasions the need to improve government procurement practices to improve government transparency: see, eg, Information and Privacy Commission (NSW), NSW IPC Fact Sheet (n 45) 2.
[208] An Act Relating to Establishing Guidelines for Government Procurement and Use of Automated Decision Systems in Order To Protect Consumers, Improve Transparency, and Create More Market Predictability, Wash S 5116, 67th Legislature (2021) §§ 4–6
(‘ADMS Procurement and Use Act’). An executive session was scheduled in relation to this proposed Act on 7 February 2022, but no action has since been taken. This Act has now lapsed: ‘Bill Information: SB 5116’, Washington State Legislature (Web Page) <https://app.leg.wa.gov/billsummary?BillNumber=5116&Year=2021&Initiative=false#documentSection>, archived at <https://perma.cc/HX5F-LWQJ>.
[209] ADMS Procurement and Use Act (n 208) § 4(5).
[210] Bloch-Wehba (n 87) 1308, discussing ‘ADMS Procurement and Use Act’ (n 208).
[211] Information and Privacy Commission (NSW), NSW IPC Fact Sheet (n 45) 2.
[212] Tydd and Gavel (n 180) 17 (emphasis in original).
[213] NSW Ombudsman, The New Machinery of Government (n 13) 62 (emphasis in original), citing Katie Miller, ‘The Application of Administrative Law Principles to Technology-Assisted Decision-Making’ (2016) 86 (December) Australian Institute of Administrative Law Forum 20, 32.
[214] ‘Algorithmic Transparency Recording Standard: Guidance for Public Sector Bodies’, Gov.UK (Web Page, 5 January 2023) <https://www.gov.uk/government/publications/guidance-for-organisations-using-the-algorithmic-transparency-recording-standard/algorithmic-transparency-recording-standard-guidance-for-public-sector-bodies>, archived at <https://perma.cc/2XFX-YNES>; ‘Algorithmic Transparency Recording Standard: Template’, Gov.UK (Web Page, 18 April 2024) <https://www.gov.uk/government/publications/algorithmic-transparency-template>, archived at <https://perma.cc/52GY-CC3G>.
[215] Proposal for a Regulation of the European Parliament and of the Council: Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act) and Amending Certain Union Legislative Acts, COM(2021) 206 final (24 April 2021) art II, annex IV (‘EU Regulation Proposal’).
[216] World Economic Forum et al, A Policy Framework for Responsible Limits of Facial Recognition: Use Case Law Enforcement Investigations (Report, November 2022) 26, 33 <https://www.weforum.org/reports/a-policy-framework-for-responsible-limits-on-facial-recognition-use-case-law-enforcement-investigations-revised-2022>, archived at <https://perma.cc/5JVM-9YQV>.
[217] See, eg, Centre for Internet and Society, Open Government: A Guide to Best Practice in Transparency, Accountability and Civic Engagement across the Public Sector (Report, 2011) 1–3 <http://www.transparencyinitiative.org/archive/wp-content/uploads/2011/09/15-Open-gover nment11.pdf> , archived at <http://perma.cc/V244-AJME> Association for Computing Machinery US Public Policy Council, ‘Statement on Algorithmic Transparency and Accountability’ (Statement, 12 January 2017) 2 <http://www.acm.org/binaries/ content/ assets/public-policy/2017_usacm_statement_algorithms.pdf> , archived at <http://perma.cc/ 5TKE-LFHT> .
[218] See, eg, Rita Matulionyte, ‘Increasing Transparency around Facial Recognition Technologies in Law Enforcement: Towards a Model Framework’ (2024) 33(1) Information and Communications Technology Law 66, 72, 75–7.
[219] Ibid 75.
[220] See, eg, NSW Government, Artificial Intelligence Assurance Framework (Report, March 2022) 13.
[221] EU Regulation Proposal (n 215) art 9.
AustLII:
Copyright Policy
|
Disclaimers
|
Privacy Policy
|
Feedback
URL: http://www.austlii.edu.au/au/journals/MelbULawRw/2024/15.html